Shuffling Data After Union and Write

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Shuffling Data After Union and Write

SNEHASISH DUTTA
Hi,

I am currently facing an issue , while performing union on three data fames say df1,df2,df3 once the operation is performed and I am trying to save the data , the data is getting shuffled so the ordering of data in df1,df2,df3 are not maintained.

When I save the data as text/csv file the content of the data gets shuffled within.
There is no way to order the dataframe as these 3 dataframes don't share any common field/constraint.

Let me know if there is a work around to maintain the ordering of the dataframes after union and write.

Regards,
Snehasish
Reply | Threaded
Open this post in threaded view
|

Re: Shuffling Data After Union and Write

Rahul Nandi
You can put a new column say order to each of the DF having 1, 2 and 3 for df1, df2 and df3 respectively. Then you can sort the data based on the order.

On Fri 13 Apr, 2018, 21:56 SNEHASISH DUTTA, <[hidden email]> wrote:
Hi,

I am currently facing an issue , while performing union on three data fames say df1,df2,df3 once the operation is performed and I am trying to save the data , the data is getting shuffled so the ordering of data in df1,df2,df3 are not maintained.

When I save the data as text/csv file the content of the data gets shuffled within.
There is no way to order the dataframe as these 3 dataframes don't share any common field/constraint.

Let me know if there is a work around to maintain the ordering of the dataframes after union and write.

Regards,
Snehasish