Executors across Stages

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Executors across Stages

AVS Bharadwaj
Hi,
I am running spark in cluster mode ( on K8 ). On running it on Word Count
example, the number of executors assigned are different across stages. Our
number of assigned executors is 20. While stage 1 gets all 20 of them
alloted, stage 2 gets only < 10 executors. Is there any particular reason
for the change in executor count across stages?



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]