Spark Structured streaming 2.4 - Kill and deploy in yarn

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Spark Structured streaming 2.4 - Kill and deploy in yarn

khajaasmath786
Hi ,

I am looking for some information on how to gracefully kill the spark structured streaming kafka job and redeploy it.

How to kill a spark structured job in YARN? 
any suggestions on how to kill gracefully?

I was able to monitor the job from SQL tab but how can I kill this job when deployed in YARN without knowing yarn id?

Thanks,
Asmath