Loop through Dataframes

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Loop through Dataframes

khajaasmath786

Hi,

What is the best approach to loop through 3 dataframes in scala based on some keys instead of using collect.

Thanks,
Asmath
Reply | Threaded
Open this post in threaded view
|

Re: Loop through Dataframes

Holden Karau
So if you want to process the contents of a dataframe locally but not pull all of the data back at once toLocaliterator is probably what you're looking for, it's still not great though so maybe you can share the root problem which your trying to solve and folks might have some suggestions there.

On Sun, Oct 6, 2019 at 2:49 PM KhajaAsmath Mohammed <[hidden email]> wrote:

Hi,

What is the best approach to loop through 3 dataframes in scala based on some keys instead of using collect.

Thanks,
Asmath


--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9