High Availability for spark streaming application running in kubernetes

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

High Availability for spark streaming application running in kubernetes

shensonj
Hello,

I have a spark streaming application running in kubernetes and we use spark operator to submit spark jobs. Any suggestion on

1. How to handle high availability for spark streaming applications.
2. What would be the best approach to handle high availability of checkpoint data if we don't use HDFS?

Thanks
Shenson