Launching EC2 instances with Spark compiled for Scala 2.11

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Launching EC2 instances with Spark compiled for Scala 2.11

Theodore Vasiloudis
This post has NOT been accepted by the mailing list yet.
Hello,

I was wondering if there is an easy way launch EC2 instances which have a Spark built for Scala 2.11.

The only way I can think of is to prepare the sources for 2.11 as shown in the Spark build instructions (http://spark.apache.org/docs/latest/building-spark.html#building-for-scala-211), upload the changed sources as a Github repo, and use the "--spark-git-repo" option to specify the repo as the one to build from.

Is there a recommended way to launch EC2 instances if you need Scala 2.11 support?

Regards,
Theodore
Reply | Threaded
Open this post in threaded view
|

Re: Launching EC2 instances with Spark compiled for Scala 2.11

Nuno Santos
This post has NOT been accepted by the mailing list yet.
Hello,

Any updates on this question? I'm also very interested in a solution, as I'm trying to use Spark on EC2 but need Scala 2.11 support. The scripts in the ec2 directory of the Spark distribution install use Scala 2.10 by default and I can't see any obvious option to change to Scala 2.11.

Regards,
Nuno