Spark 3.0.0 spark.read.json never completes

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Spark 3.0.0 spark.read.json never completes

Sanjeev Mishra
HI all,

I have huge amount of json files that Spark 2.4 can easily finish reading but Spark 3.0.0 never competes. I am running both Spark 2 and Spark 3 on Mac