Get application id when using SparkSubmit.main from java

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Get application id when using SparkSubmit.main from java

Ron Gonzalez-2
Hi,
  I am trying to get the application id after I use SparkSubmit.main for a yarn submission.
  I am able to make it asynchronous using spark.yarn.watForCompletion=false configuration option, but I can't seem to figure out how I can get the application id for this job. I read both SparkSubmit.scala and Client.scala.
  Any thoughts on how I could do it? I'd prefer not to use Client.run directly that returns the application id since there's a lot of environment prep that SparkSubmit actually does.
  Thanks in advance for any help...

Thanks,
Ron