Array and RDDs

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Array and RDDs

pradhandeep
Hi,
I have an input file which consists of <stc_node dest_node>
I have created and RDD consisting of key-value pair where key is the node id and the values are the children of that node.
Now I want to associate a byte with each node. For that I have created a byte array.
Every time I print out the key-value pair in the RDD the key-value pairs do not come in the same order. Because of this I am finding it difficult to assign the byte values with each node.
Can anyone help me out in this matter?

I basically have the following code:
    val bitarray = Array.fill[Byte](number)(0)
   
And I want to assiciate each byte in the array to a node.
How should I do that?

Thank You
Reply | Threaded
Open this post in threaded view
|

Re: Array and RDDs

Mayur Rustagi
Your question is a bit confusing.. 
I assume you have a RDD containing nodes & some meta data (child nodes maybe) & you are trying to attach another metadata to it (bye array). if its just same byte array for all nodes you can generate rdd with the count of nodes & zip the two rdd together, you can also create a (node, bytearray) combo & join the two rdd together. 


Mayur Rustagi
Ph: +1 (760) 203 3257

On Sat, Sep 6, 2014 at 10:51 AM, Deep Pradhan <[hidden email]> wrote:
Hi,
I have an input file which consists of <stc_node dest_node>
I have created and RDD consisting of key-value pair where key is the node id and the values are the children of that node.
Now I want to associate a byte with each node. For that I have created a byte array.
Every time I print out the key-value pair in the RDD the key-value pairs do not come in the same order. Because of this I am finding it difficult to assign the byte values with each node.
Can anyone help me out in this matter?

I basically have the following code:
    val bitarray = Array.fill[Byte](number)(0)
   
And I want to assiciate each byte in the array to a node.
How should I do that?

Thank You