why spark oom off-heap?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

why spark oom off-heap?

jibiyr@qq.com
hello,Why spark usually off-heap oom when shuffle reader? I read some source code , When a ResultTask read shuffle data from no-local executor,it has buffer and spill disk,so why still off-heap oom?