[SparkStreaming] How To Stop The SparkStreamingContenxt Gracefully Without Extra Time Cost?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Lyx
Reply | Threaded
Open this post in threaded view
|

[SparkStreaming] How To Stop The SparkStreamingContenxt Gracefully Without Extra Time Cost?

Lyx
hi, I've being using StreamingContext.stop(true,true) ,trying to stop my application gracefully,which means it can promise all received data will be processed before the whole application terminated. It dose works ,but I also noticed that it will also lead to extra time just waiting for empty rdds incoming ,how can I reduce the extra time waiting for empty rdds and keep my application terminated gracefully in the mean time?