DataSet save to parquet partition problem

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

DataSet save to parquet partition problem

JF Chen
I am trying to save a DataSet<Row> object to parquet file via 
df.write().partitionBy("...").parquet(path)

while this dataset contains the following struct:
time: struct 
    -dayOfMonth
    -monthOfYear
    ...

Can I use the child field like time.monthOfYear as above in partition ? If yes, how?

Thanks!

Regard,
Junfeng Chen