Spark 2.2 Structured Streaming + Kinesis

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark 2.2 Structured Streaming + Kinesis

benassi
Does anyone know if there is a connector for AWS Kinesis that can be used as a source for Structured Streaming?

Thanks.

Reply | Threaded
Open this post in threaded view
|

Re: Spark 2.2 Structured Streaming + Kinesis

Jules Damji
You can use the Databricks to connect to Kinesis: 


Cheers 
Jules 

Sent from my iPhone
Pardon the dumb thumb typos :)

On Nov 13, 2017, at 3:15 PM, Benjamin Kim <[hidden email]> wrote:

Does anyone know if there is a connector for AWS Kinesis that can be used as a source for Structured Streaming?

Thanks.

Reply | Threaded
Open this post in threaded view
|

Re: Spark 2.2 Structured Streaming + Kinesis

benassi
In reply to this post by benassi
To add, we have a CDH 5.12 cluster with Spark 2.2 in our data center.

On Mon, Nov 13, 2017 at 3:15 PM Benjamin Kim <[hidden email]> wrote:
Does anyone know if there is a connector for AWS Kinesis that can be used as a source for Structured Streaming?

Thanks.