[SQL] How to get an encoder for string array in java?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[SQL] How to get an encoder for string array in java?

tanelk
I'm trying to implement org.apache.spark.sql.expressions.Aggregator in java.

Both the input and output columns are arrays of strings. I can not figure
out how to construct a working encoder for the method outputEncoder() and
for the UDF registration. The data type on java side could be
Collection<String> or String[] or something equivalent.

Encoders.bean does not work, because array is not struct type (throws
assertion error).
Encoders.kyro does not work because the data types on spark side do not
match (array<String> vs binary)

Any help would be appreciated,
Tanel





--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]