Spark UI History server on Kubernetes

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark UI History server on Kubernetes

Battini Lakshman
Hello,

We are running Spark 2.4 on Kubernetes cluster, able to access the Spark UI using "kubectl port-forward".

However, this spark UI contains currently running Spark application logs, we would like to maintain the 'completed' spark application logs as well. Could someone help us to setup 'Spark History server' on Kubernetes. Thanks!

Best Regards,
Lakshman Battini.
Reply | Threaded
Open this post in threaded view
|

RE: Spark UI History server on Kubernetes

Rao, Abhishek (Nokia - IN/Bangalore)

Hi,

 

We’ve setup spark-history service (based on spark 2.4) on K8S. UI works perfectly fine when running on NodePort. We’re facing some issues when on ingress.

Please let us know what kind of inputs do you need?

 

Thanks and Regards,

Abhishek

 

From: Battini Lakshman <[hidden email]>
Sent: Tuesday, January 22, 2019 6:02 PM
To: [hidden email]
Subject: Spark UI History server on Kubernetes

 

Hello,

 

We are running Spark 2.4 on Kubernetes cluster, able to access the Spark UI using "kubectl port-forward".

 

However, this spark UI contains currently running Spark application logs, we would like to maintain the 'completed' spark application logs as well. Could someone help us to setup 'Spark History server' on Kubernetes. Thanks!

 

Best Regards,

Lakshman Battini.

Reply | Threaded
Open this post in threaded view
|

RE: Spark UI History server on Kubernetes

Rao, Abhishek (Nokia - IN/Bangalore)

Hi Lakshman,

 

We’ve set these 2 properties to bringup spark history server

 

spark.history.fs.logDirectory <location where the logs are stored>

spark.history.ui.port <Port on which history server should come up>

 

We’re writing the logs to HDFS. In order to write logs, we’re setting following properties while submitting the spark job

spark.eventLog.enabled true

spark.eventLog.dir <Directory where logs should be written>

 

Thanks and Regards,

Abhishek

 

From: Battini Lakshman <[hidden email]>
Sent: Wednesday, January 23, 2019 1:55 PM
To: Rao, Abhishek (Nokia - IN/Bangalore) <[hidden email]>
Subject: Re: Spark UI History server on Kubernetes

 

HI Abhishek,

 

Thank you for your response. Could you please let me know the properties you configured for bringing up History Server and its UI. 

 

Also, are you writing the logs to any directory on persistent storage, if yes, could you let me know the changes you did in Spark to write logs to that directory. Thanks!

 

Best Regards,

Lakshman Battini.

 

On Tue, Jan 22, 2019 at 10:53 PM Rao, Abhishek (Nokia - IN/Bangalore) <[hidden email]> wrote:

Hi,

 

We’ve setup spark-history service (based on spark 2.4) on K8S. UI works perfectly fine when running on NodePort. We’re facing some issues when on ingress.

Please let us know what kind of inputs do you need?

 

Thanks and Regards,

Abhishek

 

From: Battini Lakshman <[hidden email]>
Sent: Tuesday, January 22, 2019 6:02 PM
To: [hidden email]
Subject: Spark UI History server on Kubernetes

 

Hello,

 

We are running Spark 2.4 on Kubernetes cluster, able to access the Spark UI using "kubectl port-forward".

 

However, this spark UI contains currently running Spark application logs, we would like to maintain the 'completed' spark application logs as well. Could someone help us to setup 'Spark History server' on Kubernetes. Thanks!

 

Best Regards,

Lakshman Battini.

Reply | Threaded
Open this post in threaded view
|

Re: Spark UI History server on Kubernetes

Li Gao-2
In addition to what Rao mentioned, if you are using cloud blob storage such as AWS S3, you can specify your history location to be an S3 location such as:  `s3://mybucket/path/to/history`


On Wed, Jan 23, 2019 at 12:55 AM Rao, Abhishek (Nokia - IN/Bangalore) <[hidden email]> wrote:

Hi Lakshman,

 

We’ve set these 2 properties to bringup spark history server

 

spark.history.fs.logDirectory <location where the logs are stored>

spark.history.ui.port <Port on which history server should come up>

 

We’re writing the logs to HDFS. In order to write logs, we’re setting following properties while submitting the spark job

spark.eventLog.enabled true

spark.eventLog.dir <Directory where logs should be written>

 

Thanks and Regards,

Abhishek

 

From: Battini Lakshman <[hidden email]>
Sent: Wednesday, January 23, 2019 1:55 PM
To: Rao, Abhishek (Nokia - IN/Bangalore) <[hidden email]>
Subject: Re: Spark UI History server on Kubernetes

 

HI Abhishek,

 

Thank you for your response. Could you please let me know the properties you configured for bringing up History Server and its UI. 

 

Also, are you writing the logs to any directory on persistent storage, if yes, could you let me know the changes you did in Spark to write logs to that directory. Thanks!

 

Best Regards,

Lakshman Battini.

 

On Tue, Jan 22, 2019 at 10:53 PM Rao, Abhishek (Nokia - IN/Bangalore) <[hidden email]> wrote:

Hi,

 

We’ve setup spark-history service (based on spark 2.4) on K8S. UI works perfectly fine when running on NodePort. We’re facing some issues when on ingress.

Please let us know what kind of inputs do you need?

 

Thanks and Regards,

Abhishek

 

From: Battini Lakshman <[hidden email]>
Sent: Tuesday, January 22, 2019 6:02 PM
To: [hidden email]
Subject: Spark UI History server on Kubernetes

 

Hello,

 

We are running Spark 2.4 on Kubernetes cluster, able to access the Spark UI using "kubectl port-forward".

 

However, this spark UI contains currently running Spark application logs, we would like to maintain the 'completed' spark application logs as well. Could someone help us to setup 'Spark History server' on Kubernetes. Thanks!

 

Best Regards,

Lakshman Battini.