Writing to mysql from pyspark spark structured streaming

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Writing to mysql from pyspark spark structured streaming

Krishnanand Khambadkone
Hi,  I am trying to write to mysql from a spark structured streaming kafka source.  Using spark 2.4.0. I get this exception,

AnalysisException: u"'write' can not be called on streaming Dataset/DataFrame


Reply | Threaded
Open this post in threaded view
|

Re: Writing to mysql from pyspark spark structured streaming

German Schiavon Matteo
Hi, 

can you share the code?

You have to call writeStream on streaming Dataset/Dataframe

On Fri, 16 Oct 2020 at 02:14, Krishnanand Khambadkone <[hidden email]> wrote:
Hi,  I am trying to write to mysql from a spark structured streaming kafka source.  Using spark 2.4.0. I get this exception,

AnalysisException: u"'write' can not be called on streaming Dataset/DataFrame