how to add rddID to tuples in DStream

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

how to add rddID to tuples in DStream

amoc

Hi

I’ve another simple question:

I have a DStream and want to add the ID of each RDD to the tuples contained in that RDD and return a DStream of them.

So far I’ve figured out how to do this via foreachRDD and return RDDs.

How to create a DStream from these RDDs? Or even better, how to use .map to do this on the dstream like

  mystream.map( tuple => (tuple, getRDDIdSomehow) )

 

-Adrian