Livy Failed error on Yarn with Spark

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Livy Failed error on Yarn with Spark

Chetan Khatri
All,

I am running on Hortonworks HDP Hadoop with Livy and Spark 2.2.0, when I am running same spark job using spark-submit it is getting success with all transformations are done. 

When I am trying to do spark submit using Livy, at that time Spark Job is getting invoked and getting success but Yarn status says : FAILED and when you take a look on logs at attempt : Log says  SUCCESS and there is no error log.

Any one has faced this weird exprience ?

Thank you.
Reply | Threaded
Open this post in threaded view
|

Re: Livy Failed error on Yarn with Spark

zjffdu

Could you check the the spark app's yarn log and livy log ?


Chetan Khatri <[hidden email]>于2018年5月10日周四 上午4:18写道:
All,

I am running on Hortonworks HDP Hadoop with Livy and Spark 2.2.0, when I am running same spark job using spark-submit it is getting success with all transformations are done. 

When I am trying to do spark submit using Livy, at that time Spark Job is getting invoked and getting success but Yarn status says : FAILED and when you take a look on logs at attempt : Log says  SUCCESS and there is no error log.

Any one has faced this weird exprience ?

Thank you.