Spark preferred compression format

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Spark preferred compression format

Debasish Das
Hi,

What's the splittable compression format that works with Spark right now ?

We are looking into bzip2 / lzo / gzip...gzip is not splittable so not a good option....Within bzip2/lzo I am confused.

Thanks.
Deb