Shuffle Spill to Disk

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Shuffle Spill to Disk

Jack Kolokasis
Hello,

I am trying to measure how many bytes spill to disk in shuffle operation
and I get always zero. This is not correct because the spark local disk
is utilized.

Can anyone explain me why the spill counter is zero?

Thanks,
Iacovos

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]