Long-live Spark web ui

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Long-live Spark web ui

Shihaoliang (Shihaoliang)

Hi all,

 

We know that each SparkContext launches a web ui, but if we run in mesos mode, each job will initialize a SparkContext with webui.

 

What I want is a long-live spark web ui, record every job’s statistics, like history server in YARN.

 

Is there any solution to achieve this?

 

Thanks.

Peter Shi

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Long-live Spark web ui

Xia, Junluan

Hi Haoliang

 

         We are managing to implement tasks of persisting web UI, now you could refer to https://spark-project.atlassian.net/browse/SPARK-969

 

From: Shihaoliang (Shihaoliang) [mailto:[hidden email]]
Sent: Friday, December 27, 2013 9:17 AM
To: [hidden email]
Subject: Long-live Spark web ui

 

Hi all,

 

We know that each SparkContext launches a web ui, but if we run in mesos mode, each job will initialize a SparkContext with webui.

 

What I want is a long-live spark web ui, record every job’s statistics, like history server in YARN.

 

Is there any solution to achieve this?

 

Thanks.

Peter Shi

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

答复: Long-live Spark web ui

Shihaoliang (Shihaoliang)

Thanks for telling me the info,

 

When it will be done?

 

 

发件人: Xia, Junluan [mailto:[hidden email]]
发送时间: 20131227 13:57
收件人: [hidden email]
主题: RE: Long-live Spark web ui

 

Hi Haoliang

 

         We are managing to implement tasks of persisting web UI, now you could refer to https://spark-project.atlassian.net/browse/SPARK-969

 

From: Shihaoliang (Shihaoliang) [[hidden email]]
Sent: Friday, December 27, 2013 9:17 AM
To: [hidden email]
Subject: Long-live Spark web ui

 

Hi all,

 

We know that each SparkContext launches a web ui, but if we run in mesos mode, each job will initialize a SparkContext with webui.

 

What I want is a long-live spark web ui, record every job’s statistics, like history server in YARN.

 

Is there any solution to achieve this?

 

Thanks.

Peter Shi

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Long-live Spark web ui

Xia, Junluan

If everything is ready, it will be contained in next release.

 

From: Shihaoliang (Shihaoliang) [mailto:[hidden email]]
Sent: Friday, December 27, 2013 2:02 PM
To: [hidden email]
Subject:
答复: Long-live Spark web ui

 

Thanks for telling me the info,

 

When it will be done?

 

 

发件人: Xia, Junluan [[hidden email]]
发送时间: 20131227 13:57
收件人: [hidden email]
主题: RE: Long-live Spark web ui

 

Hi Haoliang

 

         We are managing to implement tasks of persisting web UI, now you could refer to https://spark-project.atlassian.net/browse/SPARK-969

 

From: Shihaoliang (Shihaoliang) [[hidden email]]
Sent: Friday, December 27, 2013 9:17 AM
To: [hidden email]
Subject: Long-live Spark web ui

 

Hi all,

 

We know that each SparkContext launches a web ui, but if we run in mesos mode, each job will initialize a SparkContext with webui.

 

What I want is a long-live spark web ui, record every job’s statistics, like history server in YARN.

 

Is there any solution to achieve this?

 

Thanks.

Peter Shi

 

Loading...