Pyspark Convert Struct Type to Map Type

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Pyspark Convert Struct Type to Map Type

anbutech
Hello Sir,

Could you please advise the below scenario in pyspark 2.4.3 in data-bricks
to load the data into the delta table.

I want to load the dataframe with this column "data" into the table as Map
type in the data-bricks spark delta table.could you please advise on this
scenario.how to convert struct type into map type

i have a column named "data" which is struct type. pls find the sample data
format:

data:struct
        field1:long
        field2:string
        field3:array
        field4:long
        field5:struct
                code:string
                name:string
        field6:struct
                lat:double
                lon:double
        field7:long
        field8:string
        field9:boolean
        field10:long
        field11:string
        field12:string
        field13:long
        field14:boolean
        field15:array
                element:string
                id:string
        field16:array
                element:string
       
Thanks
Anbu



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]