Spark UI Streaming batch time interval does not match batch interval

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Spark UI Streaming batch time interval does not match batch interval

Jordan Pilat
Hello,

I am running a streaming app on Spark 2.1.2.  The batch interval is set to 5000ms, and when I go to the "Streaming" tab in the Spark UI, it correctly reports a 5 second batch interval, but the list of batches below only shows one batch every two minutes (IE the batch time for each batch is two minutes after the prior batch).

Keep in mind, I am not referring to delay or processing time -- only when each batch starts.  What could account for this discrepancy?

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]