Access all of the custom streaming query listeners that were registered to spark session

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Access all of the custom streaming query listeners that were registered to spark session

Natalie Ruiz

Hello,

 

Is there a way to access all of the custom listeners that have been registered to a spark session? I want to remove the listeners that I am no longer using, except I don’t know what they were saved as, I just see testing output messages on my  streaming query.  I created a stack overflow question here for more detail:

 

https://stackoverflow.com/questions/57876607/how-to-access-all-of-an-apache-sparks-session-streaming-query-custom-listeners

 

Thanks you,

Natalie

Reply | Threaded
Open this post in threaded view
|

Re: Access all of the custom streaming query listeners that were registered to spark session

Gabor Somogyi
Hi Natalie,

In ideal situation it would be good to keep track of the registered listeners.
If this is not possible in 3.0 new feature added here: https://github.com/apache/spark/pull/25518

BR,
G


On Tue, Sep 10, 2019 at 10:18 PM Natalie Ruiz <[hidden email]> wrote:

Hello,

 

Is there a way to access all of the custom listeners that have been registered to a spark session? I want to remove the listeners that I am no longer using, except I don’t know what they were saved as, I just see testing output messages on my  streaming query.  I created a stack overflow question here for more detail:

 

https://stackoverflow.com/questions/57876607/how-to-access-all-of-an-apache-sparks-session-streaming-query-custom-listeners

 

Thanks you,

Natalie