I have used Spark with HDFS encrypted with Hadoop KMS, and it worked well. Somehow, I could not recall, if I had the kubernetes in the mix. Somehow, seeing the error, it is not clear what caused the failure. Can I reproduce this somehow?
On Sat, Aug 15, 2020 at 7:18 PM Michel Sumbul <[hidden email]> wrote:
I have the problem only on K8S, it's working fine when spark is executed on top of yarn.
I'm asking myself if the delegation gets saved, any idea how to check that?
Could it be because kms is in HA and spark request 2 delegation token?
For the testing, just running spark3 on top of any k8s cluster reading data to any hadoop3 with kms should be fine. I'm using a HDP3 cluster, but there is probably a more easy way to test.
Le mer. 19 août 2020 à 09:50, Prashant Sharma <[hidden email]> a écrit :
|Free forum by Nabble||Edit this page|