Union of RDDs Hung

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Union of RDDs Hung

Vikash Pareek
Hi All,

I am unioning 2 rdds(each of them having 2 records) but this union it is
getting hang.
I found a solution to this that is caching both the rdds before performing
union but I could not figure out the root cause of hanging the job.

Is somebody knows why this happens with union?

Spark version I am using is 1.6.1


Best Regards,
Vikash Pareek



-----

__Vikash Pareek
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]


__Vikash Pareek
Reply | Threaded
Open this post in threaded view
|

Re: Union of RDDs Hung

maasg
Can you show us the code?

On Tue, Dec 12, 2017 at 9:02 AM, Vikash Pareek <[hidden email]> wrote:
Hi All,

I am unioning 2 rdds(each of them having 2 records) but this union it is
getting hang.
I found a solution to this that is caching both the rdds before performing
union but I could not figure out the root cause of hanging the job.

Is somebody knows why this happens with union?

Spark version I am using is 1.6.1


Best Regards,
Vikash Pareek



-----

__Vikash Pareek
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]