Re: Unable to infer schema pf Parquet in Spark 2.0.0

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: Unable to infer schema pf Parquet in Spark 2.0.0

hemant singh
Try with path as s3:// xyz/<folder>/  ending with forward slash.

On Thu, Jul 12, 2018 at 3:49 PM, Priya Ch <[hidden email]> wrote:
Hi All, 

  I am trying to write a dataframe into S3 in PARQUET format in Overwrite mode. However, post the job succeeds, I could see the results when queried from Hive as -



When trying to read the same table in Spark, I am seeing 0 records and reading from S3 path, I see the error "Unable to infer schema for parquet." Here are the screenshots -




Please someone try to throw light on this.

Thanks, 
Padma CH