Parquet

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Parquet

amin mohebbi
We do have two big tables each includes 5 billion of rows, so my question here is should we partition /sort the data and convert it to Parquet before doing any join?

Best Regards ....................................................... Amin Mohebbi PhD candidate in Software Engineering   at university of Malaysia   Tel : +60 18 2040 017 E-Mail : [hidden email]               [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Parquet

muthu
I generally write to Parquet when I want to repeat the operation of reading data and perform different operations on it every time. This would save db time for me. 

Thanks 
Muthu 

On Thu, Jul 19, 2018, 18:34 amin mohebbi <[hidden email]> wrote:
We do have two big tables each includes 5 billion of rows, so my question here is should we partition /sort the data and convert it to Parquet before doing any join?

Best Regards ....................................................... Amin Mohebbi PhD candidate in Software Engineering   at university of Malaysia   Tel : +60 18 2040 017 E-Mail : [hidden email]               [hidden email]