[SparkContext] will application immediately stop after sc.stop()?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[SparkContext] will application immediately stop after sc.stop()?

bsikander
Is it possible that a job keeps on running for some time after
onApplicationEnd is fired?

For example,
I have a spark job which has 10 batches still to process and let's say that
the processing them will take 10 minutes. If I execute sparkContext.stop(),
I will receive onApplicationEnd immediately according to  this code
<https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L1902>
. But after receiving the event, will my spark job still continue to run for
10 minutes and finish the batch execution?



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]