modeling timestamp in Avro messages (read using Spark Structured Streaming)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

modeling timestamp in Avro messages (read using Spark Structured Streaming)

karan alang
i've a questing regarding modeling timestamp column in Avro messages
The options are
- ISO 8601 "String" (UTC Time)
- "int" 32bit signed UNIX Epoch time
- Long (modeled as Logica datatype - timestamp in schema)
what would be the best way to model the timestamp ?

fyi. we are using Apache Spark(Structured Streaming) to read these messages from kafka topics.