Getting FileNotFoundException and LeaseExpired Exception while writing a df to hdfs path

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Getting FileNotFoundException and LeaseExpired Exception while writing a df to hdfs path

Gaurav Gupta
Hi

I am receiving FileNotFoundException and LeaseExpired Exception while
writing a data frame to an hdfs path.I am using spark 1.6 and reading
messages from Tibco in my streaming application .I am doing some
tranformations on each Rdd and converting it to a data frame and writing to
an hdfs path.

I am using spark.speculation=false as well but I am still receiving this
error.Please help as this a issue in my prod environment.
Here is the link to my detailed question on stack overflow

https://stackoverflow.com/questions/53911624/getting-java-io-filenotfoundexception-while-saving-a-dataframe-to-hdfs-path

Regards
Gaurav Gupta

 



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]