Spark 2.1.0 hanging while writing a table in HDFS in parquet format

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Spark 2.1.0 hanging while writing a table in HDFS in parquet format

gae123
This post has NOT been accepted by the mailing list yet.
I have put more details and stack traces here:

http://stackoverflow.com/questions/43462638/emr-spark-2-1-0-process-get-stuck-at-at-org-apache-spark-unsafe-platform-copymem

Any suggestions would be very much appreciated.