Way to get the file name of the output when doing ORC write from dataframe

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Way to get the file name of the output when doing ORC write from dataframe

Manjunath Shetty H
Hi,

I wanted to know if there is any way to get the output file name that `Dataframe.orc()` will write to ?. This is needed to track which file is written by which job during incremental batch jobs.

Thanks
Manjunath
Reply | Threaded
Open this post in threaded view
|

Re: Way to get the file name of the output when doing ORC write from dataframe

Manjunath Shetty H
Or is there any way to provide a Unique file name to the ORC write function itself ?

Any suggestions will be helpful.

Regards
Manjunath Shetty

From: Manjunath Shetty H <[hidden email]>
Sent: Wednesday, March 4, 2020 2:28 PM
To: user <[hidden email]>
Subject: Way to get the file name of the output when doing ORC write from dataframe
 
Hi,

I wanted to know if there is any way to get the output file name that `Dataframe.orc()` will write to ?. This is needed to track which file is written by which job during incremental batch jobs.

Thanks
Manjunath