Spark Kubernetes 3.0.1 | podcreationTimeout not working

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark Kubernetes 3.0.1 | podcreationTimeout not working

Ranju Jain

Hi,

 

I submitted the spark job and pods goes in Pending state because of insufficient resources.

But they are not getting deleted after this timeout of 60 sec. Please help me in understanding.

 

Regards

Ranju

Reply | Threaded
Open this post in threaded view
|

Re: Spark Kubernetes 3.0.1 | podcreationTimeout not working

Jacek Laskowski
Hi Ranju,

Can you show the pods and their state? Does the situation happen at the very beginning of spark-submit or some time later (once you've got a couple of executors)? My understanding allows me to think of the driver not starting up due to lack of resources or executors. In either case they're not deleted as they simply wait forever. I might be mistaken here though.

What property is this for "this timeout of 60 sec."?

On Wed, Feb 10, 2021 at 2:03 PM Ranju Jain <[hidden email]> wrote:

Hi,

 

I submitted the spark job and pods goes in Pending state because of insufficient resources.

But they are not getting deleted after this timeout of 60 sec. Please help me in understanding.

 

Regards

Ranju

Reply | Threaded
Open this post in threaded view
|

Re: Spark Kubernetes 3.0.1 | podcreationTimeout not working

Attila Zsolt Piros
I believe this problem led to opening SPARK-34389 where the problem is
discussed further.

[1] https://issues.apache.org/jira/browse/SPARK-34389



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]