Shuffle Spill to Disk

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Shuffle Spill to Disk

Jack Kolokasis

I am trying to measure how many bytes spill to disk in shuffle operation
and I get always zero. This is not correct because the spark local disk
is utilized.

Can anyone explain me why the spill counter is zero?


To unsubscribe e-mail: [hidden email]