Spark onApplicationEnd run multiple times during the application failure

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark onApplicationEnd run multiple times during the application failure

Jiang, Yi J (CWM-NR)

Hello

We are running into an issue.

We have customized the SparkListener class, and added that to spark context. But when the spark job is failed, we find the “onApplicationEnd” function was triggered twice.

Is that supposed to be triggered just once when the spark job is failed? Because the application level of spark job is only launched once, how can it be triggered twice when it is failed.

Please let us know

Thank you

 

 

_______________________________________________________________________

This email is intended only for the use of the individual(s) to whom it is addressed and may be privileged and confidential.
Unauthorised use or disclosure is prohibited. If you receive this e-mail in error, please advise immediately and delete the original message.
This message may have been altered without your or our knowledge and the sender does not accept any liability for any errors or omissions in the message.

Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent.
Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite.
Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de courriel ou par un autre moyen.

Reply | Threaded
Open this post in threaded view
|

Re: Spark onApplicationEnd run multiple times during the application failure

hemant singh
Could it be because of re-try.

Thanks

On Thu, 21 Nov 2019 at 3:35 AM, Jiang, Yi J (CWM-NR) <[hidden email]> wrote:

Hello

We are running into an issue.

We have customized the SparkListener class, and added that to spark context. But when the spark job is failed, we find the “onApplicationEnd” function was triggered twice.

Is that supposed to be triggered just once when the spark job is failed? Because the application level of spark job is only launched once, how can it be triggered twice when it is failed.

Please let us know

Thank you

 

 

_______________________________________________________________________

This email is intended only for the use of the individual(s) to whom it is addressed and may be privileged and confidential.
Unauthorised use or disclosure is prohibited. If you receive this e-mail in error, please advise immediately and delete the original message.
This message may have been altered without your or our knowledge and the sender does not accept any liability for any errors or omissions in the message.

Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent.
Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite.
Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de courriel ou par un autre moyen.

Reply | Threaded
Open this post in threaded view
|

RE: Spark onApplicationEnd run multiple times during the application failure

Jiang, Yi J (CWM-NR)

Hello,

Thank you for replying, it is retry, but why retry is happened from whole application level?

To my understanding, the retry can be done in job level.

Jacky

 

 

From: hemant singh [mailto:[hidden email]]
Sent: November 21, 19 3:12 AM
To: Jiang, Yi J (CWM-NR) <[hidden email]>
Cc: Martin, Phil <[hidden email]>; [hidden email]
Subject: Re: Spark onApplicationEnd run multiple times during the application failure

 

Could it be because of re-try.

 

Thanks

 

On Thu, 21 Nov 2019 at 3:35 AM, Jiang, Yi J (CWM-NR) <[hidden email]> wrote:

Hello

We are running into an issue.

We have customized the SparkListener class, and added that to spark context. But when the spark job is failed, we find the “onApplicationEnd” function was triggered twice.

Is that supposed to be triggered just once when the spark job is failed? Because the application level of spark job is only launched once, how can it be triggered twice when it is failed.

Please let us know

Thank you

 

 

_______________________________________________________________________

This email is intended only for the use of the individual(s) to whom it is addressed and may be privileged and confidential.
Unauthorised use or disclosure is prohibited. If you receive this e-mail in error, please advise immediately and delete the original message.
This message may have been altered without your or our knowledge and the sender does not accept any liability for any errors or omissions in the message.

Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent.
Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite.
Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de courriel ou par un autre moyen.

Reply | Threaded
Open this post in threaded view
|

Re: Spark onApplicationEnd run multiple times during the application failure

hemant singh
This is how it is. It is the whole application level retry after 4 tasks attempts fail the whole application fails and then the application retry.

Thanks,
Hemant

On Thu, 21 Nov 2019 at 7:24 PM, Jiang, Yi J (CWM-NR) <[hidden email]> wrote:

Hello,

Thank you for replying, it is retry, but why retry is happened from whole application level?

To my understanding, the retry can be done in job level.

Jacky

 

 

From: hemant singh [mailto:[hidden email]]
Sent: November 21, 19 3:12 AM
To: Jiang, Yi J (CWM-NR) <[hidden email]>
Cc: Martin, Phil <[hidden email]>; [hidden email]
Subject: Re: Spark onApplicationEnd run multiple times during the application failure

 

Could it be because of re-try.

 

Thanks

 

On Thu, 21 Nov 2019 at 3:35 AM, Jiang, Yi J (CWM-NR) <[hidden email]> wrote:

Hello

We are running into an issue.

We have customized the SparkListener class, and added that to spark context. But when the spark job is failed, we find the “onApplicationEnd” function was triggered twice.

Is that supposed to be triggered just once when the spark job is failed? Because the application level of spark job is only launched once, how can it be triggered twice when it is failed.

Please let us know

Thank you

 

 

_______________________________________________________________________

This email is intended only for the use of the individual(s) to whom it is addressed and may be privileged and confidential.
Unauthorised use or disclosure is prohibited. If you receive this e-mail in error, please advise immediately and delete the original message.
This message may have been altered without your or our knowledge and the sender does not accept any liability for any errors or omissions in the message.

Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent.
Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite.
Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de courriel ou par un autre moyen.