Why do I see five attempts on my Spark application

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Why do I see five attempts on my Spark application

noppanit
Hi, 

I'm wondering why am I seeing 5 attempts for my Spark application? Does Spark application restart itself?Screen Shot 2017-12-13 at 2.18.03 PM.png
Reply | Threaded
Open this post in threaded view
|

Re: Why do I see five attempts on my Spark application

Marcelo Vanzin
On Wed, Dec 13, 2017 at 11:21 AM, Toy <[hidden email]> wrote:
> I'm wondering why am I seeing 5 attempts for my Spark application? Does Spark application restart itself?

It restarts itself if it fails (up to a limit that can be configured
either per Spark application or globally in YARN).


--
Marcelo

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why do I see five attempts on my Spark application

noppanit
Hi,

Can you point me to the config for that please? 

On Wed, 13 Dec 2017 at 14:23 Marcelo Vanzin <[hidden email]> wrote:
On Wed, Dec 13, 2017 at 11:21 AM, Toy <[hidden email]> wrote:
> I'm wondering why am I seeing 5 attempts for my Spark application? Does Spark application restart itself?

It restarts itself if it fails (up to a limit that can be configured
either per Spark application or globally in YARN).


--
Marcelo
Reply | Threaded
Open this post in threaded view
|

Re: Why do I see five attempts on my Spark application

Subhash Sriram

On Wed, Dec 13, 2017 at 2:31 PM, Toy <[hidden email]> wrote:
Hi,

Can you point me to the config for that please? 

On Wed, 13 Dec 2017 at 14:23 Marcelo Vanzin <[hidden email]> wrote:
On Wed, Dec 13, 2017 at 11:21 AM, Toy <[hidden email]> wrote:
> I'm wondering why am I seeing 5 attempts for my Spark application? Does Spark application restart itself?

It restarts itself if it fails (up to a limit that can be configured
either per Spark application or globally in YARN).


--
Marcelo

Reply | Threaded
Open this post in threaded view
|

Re: Why do I see five attempts on my Spark application

Subhash Sriram
There are some more properties specifically for YARN here:


Thanks,
Subhash

On Wed, Dec 13, 2017 at 2:32 PM, Subhash Sriram <[hidden email]> wrote:

On Wed, Dec 13, 2017 at 2:31 PM, Toy <[hidden email]> wrote:
Hi,

Can you point me to the config for that please? 

On Wed, 13 Dec 2017 at 14:23 Marcelo Vanzin <[hidden email]> wrote:
On Wed, Dec 13, 2017 at 11:21 AM, Toy <[hidden email]> wrote:
> I'm wondering why am I seeing 5 attempts for my Spark application? Does Spark application restart itself?

It restarts itself if it fails (up to a limit that can be configured
either per Spark application or globally in YARN).


--
Marcelo


Reply | Threaded
Open this post in threaded view
|

Re: Why do I see five attempts on my Spark application

sanat kumar Patnaik
It should be within your yarn-site.xml config file.The parameter name is yarn.resourcemanager.am.max-attempts.

The directory should be /usr/lib/spark/conf/yarn-conf. Try to find this directory on your gateway node if using Cloudera distribution.

On Wed, Dec 13, 2017 at 2:33 PM, Subhash Sriram <[hidden email]> wrote:
There are some more properties specifically for YARN here:


Thanks,
Subhash

On Wed, Dec 13, 2017 at 2:32 PM, Subhash Sriram <[hidden email]> wrote:

On Wed, Dec 13, 2017 at 2:31 PM, Toy <[hidden email]> wrote:
Hi,

Can you point me to the config for that please? 

On Wed, 13 Dec 2017 at 14:23 Marcelo Vanzin <[hidden email]> wrote:
On Wed, Dec 13, 2017 at 11:21 AM, Toy <[hidden email]> wrote:
> I'm wondering why am I seeing 5 attempts for my Spark application? Does Spark application restart itself?

It restarts itself if it fails (up to a limit that can be configured
either per Spark application or globally in YARN).


--
Marcelo





--
Regards,
Sanat Patnaik
Cell->804-882-6424