Spark Structured Streaming how to read data from AWS SQS

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Spark Structured Streaming how to read data from AWS SQS

Bogdan Cojocar

For spark streaming there are connectors that can achieve this functionality.

Unfortunately for spark structured streaming I couldn't find any as it's a newer technology. Is there a way to connect to a source using a spark streaming connector? Or is there a way to create a custom connector similar to the way one can be created in a spark streaming application?


Many thanks,

Bogdan Cojocar