Spark Thrift Server in Kubernetes deployment

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark Thrift Server in Kubernetes deployment

Subash K

Hi,

 

We are currently using Spark 2.4.4 with Spark Thrift Server (STS) to expose a JDBC interface to the reporting tools to generate report from Spark tables.

 

Now as we are analyzing on containerized deployment of Spark and STS, I would like to understand is STS deployment on Kubernetes is supported out of the box? Because we were not able to find any document link on how to configure and spin up the container for STS. Please help us on this.

 

Regards,

Subash Kunjupillai

 

Reply | Threaded
Open this post in threaded view
|

RE: Spark Thrift Server in Kubernetes deployment

Rao, Abhishek (Nokia - IN/Bangalore)

Hi,

 

STS deployment on k8s is not supported out of the box.

We had done some minor changes in spark code to get Spark Thrift Server working on k8s.

Here is the PR that we had created.

https://github.com/apache/spark/pull/22433

 

Unfortunately, this could not be merged.

 

Thanks and Regards,

Abhishek

 

From: Subash K <[hidden email]>
Sent: Monday, June 22, 2020 9:00 AM
To: [hidden email]
Subject: Spark Thrift Server in Kubernetes deployment

 

Hi,

 

We are currently using Spark 2.4.4 with Spark Thrift Server (STS) to expose a JDBC interface to the reporting tools to generate report from Spark tables.

 

Now as we are analyzing on containerized deployment of Spark and STS, I would like to understand is STS deployment on Kubernetes is supported out of the box? Because we were not able to find any document link on how to configure and spin up the container for STS. Please help us on this.

 

Regards,

Subash Kunjupillai