Can we convert scala.collection.ArrayBuffer[(Int,Double)] to org.spark.RDD[(Int,Double])

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Can we convert scala.collection.ArrayBuffer[(Int,Double)] to org.spark.RDD[(Int,Double])

yh18190
Hi,

Can we convert directly scala collection to spark RDD data type without using parellize method?
Is their any way to create custom converted RDD datatype from scala type using some typecast like that?

Please suggest me....
Reply | Threaded
Open this post in threaded view
|

Re: Can we convert scala.collection.ArrayBuffer[(Int,Double)] to org.spark.RDD[(Int,Double])

Mayur Rustagi
The scala object needs to be sent to workers to be used as a RDD, parallalize is a way to do that. What are you looking to do?
You can serialize the scala object to hdfs/disk & load it from thr 
Regards
Mayur

Mayur Rustagi
Ph: +1 (760) 203 3257


On Sun, Mar 30, 2014 at 6:22 AM, yh18190 <[hidden email]> wrote:
Hi,

Can we convert directly scala collection to spark RDD data type without
using parellize method?
Is their any way to create custom converted RDD datatype from scala type
using some typecast like that?

Please suggest me....



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Can-we-convert-scala-collection-ArrayBuffer-Int-Double-to-org-spark-RDD-Int-Double-tp3486.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.