Required fields in Parquet

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Required fields in Parquet

Dan Osipov
Hello,

Is it possible to set fields as required when writing a DataFrame/DataSet from Spark? I wasn't able to find a way to enforce a schema, and the default schema used has a lot of fields defined as "optional", while I'd like to set them to be "required".

Thanks!