executor idle during task schedule

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

executor idle during task schedule

wxhsdp
Hi, all
  i've observed that sometimes when the executor finishes one task, it will wait about 5 seconds to
  get another task to work, during the 5 seconds, the executor does nothing: cpu idle, no disk access,
  no network transfer. is that normal for spark?

  thanks!