static dataframe to streaming

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

static dataframe to streaming

aka.fe2s
Hi All,

What is the most efficient way of converting static dataframe to streaming (structured streaming)?
I have a custom sink implemented for structured streaming and I would like to use it to write a static dataframe.
I know that I can write a dataframe to files and then source them to a create a stream. Is there any more efficient way to do this?

--
Oleksiy