Trying to run Spark on Yarn

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Trying to run Spark on Yarn

I'm running into an authentication issue when running against YARN.  I am using my own method to create the JAR assembly file and most likely I am missing something.  This method used to work, but I recently ran into this problem.  Here is the error from the YARN server:

14/05/23 19:03:02 INFO yarn.WorkerLauncher: ApplicationAttemptId: appattempt_1400198337128_0012_000001
14/05/23 19:03:02 INFO yarn.WorkerLauncher: Registering the ApplicationMaster
14/05/23 19:03:02 ERROR security.UserGroupInformation: PriviledgedActionException as:zsterone (auth:SIMPLE) Client cannot authenticate via:[TOKEN]
14/05/23 19:03:02 WARN ipc.Client: Exception encountered while connecting to the server : Client cannot authenticate via:[TOKEN]

What are the obvious culprit dependencies for this issue.  I am running Spark 0.9.1 with Hadoop 2.2