[K8S] Option to keep the executor pods after job finishes

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[K8S] Option to keep the executor pods after job finishes

Li Gao-2
Hi,

Is there an option to keep the executor pods on k8s after the job finishes? We want to extract the logs and stats before removing the executor pods.

Thanks,
Li
Reply | Threaded
Open this post in threaded view
|

Re: [K8S] Option to keep the executor pods after job finishes

Yinan Li
There is currently no such an option. But this has been raised before in https://issues.apache.org/jira/browse/SPARK-25515.
 

On Tue, Oct 9, 2018 at 2:17 PM Li Gao <[hidden email]> wrote:
Hi,

Is there an option to keep the executor pods on k8s after the job finishes? We want to extract the logs and stats before removing the executor pods.

Thanks,
Li