How does order work in Row objects when .toDF() is called?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

How does order work in Row objects when .toDF() is called?

Daniel Stojanov
 >>> row_1 = psq.Row(first=1, second=2)
 >>> row_2 = psq.Row(second=22, first=11)
 >>> spark.sparkContext.parallelize([row_1, row_2]).toDF().collect()
[Row(first=1, second=2), Row(first=22, second=11)]


(Spark 3.0.1)

What is happening in the above? When .toDF() is called it appears that
order is more important than label. Are labels just a shorthand for _1,
_2, _3... and don't actually map to a particular value as they would in
a dict?




---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]