Re: [Spark SQL]: Does Spark SQL 2.3+ suppor UDT?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: [Spark SQL]: Does Spark SQL 2.3+ suppor UDT?

Suny Tyagi

Thanks and Regards,
Suny Tyagi
Phone No : 9885027192


On Mon, Nov 26, 2018 at 10:31 PM Suny Tyagi <[hidden email]> wrote:

Hi Team,


I was going through this ticket https://issues.apache.org/jira/browse/SPARK-7768?jql=text%20~%20%22public%20udt%22 and could not understand that if spark support UDT in  2.3+ version in any language (scala, python , java, R) ?

I have class something like this

Class Test{        

        string name;

        int age;

}


And My UDF method is:

public Test UDFMethod(string name, int age){           

           Test ob = new Test();

           ob.name = name;           

           ob.age = age; 

}

Sample Spark query-   `Select *, UDFMethod(name, age) From SomeTable;`

Now UDFMethod(name, age) will return Class Test object. So will this work in Spark SQL after using SQLUserDefinedType tag and extending UserDefinedType class. 

 As UserDefinedType is private in Spark 2.0. I am just want to know if UDT is support in Spark 2.3+. If yes what is the best to use UserDefinedType or UDTRegisteration. As of now both are private.


Thanks,

Suny Tyagi

 



Thanks and Regards,
Suny Tyagi
Phone No : 9885027192