In a Structured Streaming job, the listener that is supported is StreamingQueryListener.
spark.streams().addListener(
new StreamingQueryListener() {
...
}
);
However, there is no straightforward way to use StreamingListener.
I have done it like this:
StreamingContext streamingContext = new StreamingContext(spark.sparkContext(), new Duration(1000))
streamingContext.addStreamingListener(
new StreamingListener() {
...
}
)
However, this is not working for me. Is there a way to use StreamingListener in a Structured Streaming query?
Thanks,
Arwin