Integrating multiple streaming sources

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Integrating multiple streaming sources

Artemis User
Is there anyway to integrate/fuse multiple streaming sources into a
single stream process?  In other words, the current structured streaming
API dictates a single a streaming source and sink.  We'd like to have a
stream process that interfaces with multiple stream sources, perform a
join and direct the result to a single sink. Is this possible?  Thanks!

-- ND


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Integrating multiple streaming sources

Artemis User
Hmm, looks like Spark 2.3+ does support stream-to-stream join. But the
online doc doesn't provide any examples.  If anyone could provide some
concrete reference, I'd really appreciate.  Thanks! -- ND

On 12/22/20 9:57 AM, Artemis User wrote:

> Is there anyway to integrate/fuse multiple streaming sources into a
> single stream process?  In other words, the current structured
> streaming API dictates a single a streaming source and sink.  We'd
> like to have a stream process that interfaces with multiple stream
> sources, perform a join and direct the result to a single sink. Is
> this possible?  Thanks!
>
> -- ND
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]