Using Streaming Listener in a Structured Streaming job

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Using Streaming Listener in a Structured Streaming job

Arwin Tio
In a Structured Streaming job, the listener that is supported is StreamingQueryListener.

spark.streams().addListener(
  new StreamingQueryListener() {
    ...
  }
);

However, there is no straightforward way to use StreamingListener.

I have done it like this:

StreamingContext streamingContext = new StreamingContext(spark.sparkContext(), new Duration(1000))

streamingContext.addStreamingListener(
  new StreamingListener() {
    ...
  }
)
However, this is not working for me.  Is there a way to use StreamingListener in a Structured Streaming query?

Thanks,

Arwin