exception while running pi example on yarn cluster

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

exception while running pi example on yarn cluster

Venkata siva kamesh Bhallamudi
Hi All,
 I am new to Spark and running pi example on Yarn Cluster. I am getting the following exception

Exception in thread "main" java.lang.NullPointerException
    at scala.collection.mutable.ArrayOps$ofRef$.length$extension(ArrayOps.scala:114)
    at scala.collection.mutable.ArrayOps$ofRef.length(ArrayOps.scala:114)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:32)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
    at org.apache.spark.deploy.yarn.Client$.populateClasspath(Client.scala:518)
    at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:333)
    at org.apache.spark.deploy.yarn.Client.runApp(Client.scala:94)
    at org.apache.spark.deploy.yarn.Client.run(Client.scala:115)
    at org.apache.spark.deploy.yarn.Client$.main(Client.scala:492)
    at org.apache.spark.deploy.yarn.Client.main(Client.scala)

I am using
Spark Version : 0.9.0
Yarn Version : 2.3.0

Please help me where am I doing wrong.

Thanks & Regards
Kamesh.
Reply | Threaded
Open this post in threaded view
|

Re: exception while running pi example on yarn cluster

Sandy Ryza
I think this is caused by not setting yarn.application.classpath in your yarn-site.xml.

-Sandy


On Sat, Mar 8, 2014 at 2:24 AM, Venkata siva kamesh Bhallamudi <[hidden email]> wrote:
Hi All,
 I am new to Spark and running pi example on Yarn Cluster. I am getting the following exception

Exception in thread "main" java.lang.NullPointerException
    at scala.collection.mutable.ArrayOps$ofRef$.length$extension(ArrayOps.scala:114)
    at scala.collection.mutable.ArrayOps$ofRef.length(ArrayOps.scala:114)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:32)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
    at org.apache.spark.deploy.yarn.Client$.populateClasspath(Client.scala:518)
    at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:333)
    at org.apache.spark.deploy.yarn.Client.runApp(Client.scala:94)
    at org.apache.spark.deploy.yarn.Client.run(Client.scala:115)
    at org.apache.spark.deploy.yarn.Client$.main(Client.scala:492)
    at org.apache.spark.deploy.yarn.Client.main(Client.scala)

I am using
Spark Version : 0.9.0
Yarn Version : 2.3.0

Please help me where am I doing wrong.

Thanks & Regards
Kamesh.