help

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

help

Mina
This post has NOT been accepted by the mailing list yet.
I need help with spark cluster?
I have posted many posted but I haven't gotten any solution to my problem.

http://apache-spark-user-list.1001560.n3.nabble.com/what-is-the-difference-between-action-and-transformation-td3509.html

Counting action is taking way too long to perform. I suspect that there is something wrong with my spark cluster configuration. I need someone who can help me on this please.

And I was wondering if the following configuration is correct:


Thank you
Reply | Threaded
Open this post in threaded view
|

Re: help

Dhimant
This post has NOT been accepted by the mailing list yet.
This post was updated on .
try this
val conf = new SparkConf().setMaster("spark://master:7077").setAppName("Spark shell").set("spark.executor.memory", "15g")
val sc =  new SparkContext(conf)