Pyspark create RDD of dictionary

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Pyspark create RDD of dictionary

Soheil Pourbafrani
Hi, I have an RDD of the form (((a), (b), (c), (d)), (e)) and I want to transform every row to a dictionary of the form a:(b, c, d, e)

Here is my code, but it's errorful!
map(lambda row : {row[0][0] : (row[1], row[0][1], row[0][2], row[0][3]))
Is it possible to do such a transformation?


Reply | Threaded
Open this post in threaded view
|

Re: Pyspark create RDD of dictionary

Eike von Seggern
Hi,

Soheil Pourbafrani <[hidden email]> schrieb am Fr., 2. Nov. 2018 um 15:43 Uhr:
Hi, I have an RDD of the form (((a), (b), (c), (d)), (e)) and I want to transform every row to a dictionary of the form a:(b, c, d, e)

Here is my code, but it's errorful!
map(lambda row : {row[0][0] : (row[1], row[0][1], row[0][2], row[0][3]))
You're missing a `}`:

lambda row: {row[0][0]: (...)}

HTH

Eike
Reply | Threaded
Open this post in threaded view
|

Re: Pyspark create RDD of dictionary

Soheil Pourbafrani
Got it, thanks! 

On Fri, Nov 2, 2018 at 7:18 PM Eike von Seggern <[hidden email]> wrote:
Hi,

Soheil Pourbafrani <[hidden email]> schrieb am Fr., 2. Nov. 2018 um 15:43 Uhr:
Hi, I have an RDD of the form (((a), (b), (c), (d)), (e)) and I want to transform every row to a dictionary of the form a:(b, c, d, e)

Here is my code, but it's errorful!
map(lambda row : {row[0][0] : (row[1], row[0][1], row[0][2], row[0][3]))
You're missing a `}`:

lambda row: {row[0][0]: (...)}

HTH

Eike