Spark streaming event listener handling

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Spark streaming event listener handling

ArunDasM
This post has NOT been accepted by the mailing list yet.
Hi,

I am registering the spark streaming event listener handling using

jssc.addStreamingListener(new JobListener());

and i have implemented all the methods in the listener. But in these methods onBatchCompleted, onBatchStarted, onBatchSubmitted is working correctly. onReceiverError, onReceiverStarted, onReceiverStopped is not calling when killing the streaming job. JobListener is my class which implemented the StreamingListener interface. Can anyone explain what may be the problem.


import org.apache.spark.streaming.scheduler.StreamingListener;
import org.apache.spark.streaming.scheduler.StreamingListenerBatchCompleted;
import org.apache.spark.streaming.scheduler.StreamingListenerBatchStarted;
import org.apache.spark.streaming.scheduler.StreamingListenerBatchSubmitted;
import org.apache.spark.streaming.scheduler.StreamingListenerReceiverError;
import org.apache.spark.streaming.scheduler.StreamingListenerReceiverStarted;
import org.apache.spark.streaming.scheduler.StreamingListenerReceiverStopped;
class JobListener implements StreamingListener{

        @Override
        public void onBatchCompleted(StreamingListenerBatchCompleted arg0) {
                System.out.println("BatchCompleted");
        }

        @Override
        public void onBatchStarted(StreamingListenerBatchStarted arg0) {
                System.out.println("BatchStarted");
        }

        @Override
        public void onBatchSubmitted(StreamingListenerBatchSubmitted arg0) {
                System.out.println("BatchSubmitted");
        }

        @Override
        public void onReceiverError(StreamingListenerReceiverError arg0) {
                System.out.println("ReceiverError");
        }

        @Override
        public void onReceiverStarted(StreamingListenerReceiverStarted arg0) {
                System.out.println("ReceiverStarted");
        }

        @Override
        public void onReceiverStopped(StreamingListenerReceiverStopped arg0) {
                System.out.println("ReceiverStopped");
        }
       
}
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Spark streaming event listener handling

Mudasser78
This post has NOT been accepted by the mailing list yet.
Hi ArunDasM,

Im also having the same issue. If you have found the solution already please let me know. Im struck on this for a week now.

Thanks you.
Loading...