Quantcast

Text file and shuffle

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Text file and shuffle – Hi, I'm new to spark and I wanted to understand a few things conceptually so that I can optimize my spark job. I have a large text file (~14G, ...
I think the shuffle is unavoidable given that the input partitions (probably hadoop input spits in your case) are not arranged in the way of a c...
Loading...