ERROR LiveListenerBus

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

ERROR LiveListenerBus

This post has NOT been accepted by the mailing list yet.

I am running simple code that performs regex on some big file and caches the result (which is small and manageable). However, I keep getting this error when I perform a counting operation:

ERROR LiveListenerBus: Dropping SparkListenerEvent because no remaining room in event queue. This likely means one of the SparkListeners is too slow and cannot keep up withthe rate at which tasks are being started by the scheduler.

It still gives me the total count though.

Then, when I want to display the first entry (using "rdd.take(1)"), the job crashes with a different error message:

ERROR DAGSchedulerActorSupervisor: eventProcesserActor failed due to the error Cannot assign requested address; shutting down SparkContext