[SparkContext] will application immediately stop after sc.stop()?

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[SparkContext] will application immediately stop after sc.stop()?

Is it possible that a job keeps on running for some time after
onApplicationEnd is fired?

For example,
I have a spark job which has 10 batches still to process and let's say that
the processing them will take 10 minutes. If I execute sparkContext.stop(),
I will receive onApplicationEnd immediately according to  this code
. But after receiving the event, will my spark job still continue to run for
10 minutes and finish the batch execution?

Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

To unsubscribe e-mail: [hidden email]