What's the deal with --proxy-user?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

What's the deal with --proxy-user?

Jeff Evans
Hi all,

I'm trying to understand if the --proxy-user parameter to spark-submit is deprecated, or something similar?  The reason I ask is because it's hard to find documentation really talking about it.  The Spark Security doc doesn't mention it at all, nor does the YARN doc.  It does show up in the source code in this README file.  Is that supposed to be the definitive documentation for the option?

Looking into vendor specific docs, it disappears between CDH 5.9 and CDH 5.10 docs.  I couldn't find it anywhere in the HDP documentation, but I'm not as familiar with navigating there.

And perhaps I'm reading a bit too much into the tea leaves here, but the comments on SPARK-13478 (which removed the ability to specify --proxy-user along with --principal and --keytab) seem to read like "you shouldn't be using it; you should just use principal and keytab".

So, am I going crazy or is this really not clearly documented?  Is it a deprecated feature or will it continue to be supported long term?  Any kind of insight is appreciated.