Writing RDDs to HDFS is empty

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Writing RDDs to HDFS is empty

Jian Lee
Hi all,
In  my experiment program,I used spark Graphx,
when running on the Idea in windows,the result is right,
but when runing  on the linux distributed cluster,the result in hdfs is
empty,
why?how to solve?
<http://apache-spark-user-list.1001560.n3.nabble.com/file/t9811/%E9%97%AE%E9%A2%98.jpg>

Thanks!
Jian Li



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re:Writing RDDs to HDFS is empty

Yeikel
Please share a minimum amount of code to try reproduce the issue...


---- On Mon, 07 Jan 2019 00:46:42 -0800 [hidden email] wrote ----

Hi all,
In my experiment program,I used spark Graphx,
when running on the Idea in windows,the result is right,
but when runing on the linux distributed cluster,the result in hdfs is
empty,
why?how to solve?
<http://apache-spark-user-list.1001560.n3.nabble.com/file/t9811/%E9%97%AE%E9%A2%98.jpg>

Thanks!
Jian Li



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Re:Writing RDDs to HDFS is empty

Jian Lee
Sorry,the code is too long,it is simple to say
look at the photo
<http://apache-spark-user-list.1001560.n3.nabble.com/file/t9811/%E9%97%AE%E9%A2%98%E5%9B%9E%E7%AD%94.png>

i define a arrayBuffer ,there are "1 2",  '' 2 3" ," 4 5" in it ,I want to
save in hdfs ,so i make it to RDD,
sc. pallelize(arraybuffeer)
but when in idea,i use println(_),the value is right,but in distributed
there is nothing



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re:Writing RDDs to HDFS is empty

Yeikel
Ideally...we would like to copy paste and try in our end. A screenshot is not enough.

If you have private information just remove and create a minimum example we can use to replicate the issue.

I'd say similar to this :

https://stackoverflow.com/help/mcve

---- On Mon, 07 Jan 2019 04:15:16 -0800 [hidden email] wrote ----

Sorry,the code is too long,it is simple to say
look at the photo
<http://apache-spark-user-list.1001560.n3.nabble.com/file/t9811/%E9%97%AE%E9%A2%98%E5%9B%9E%E7%AD%94.png>

i define a arrayBuffer ,there are "1 2", '' 2 3" ," 4 5" in it ,I want to
save in hdfs ,so i make it to RDD,
sc. pallelize(arraybuffeer)
but when in idea,i use println(_),the value is right,but in distributed
there is nothing



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]