Using G1GC in Spark

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Using G1GC in Spark

Aakash Basu-2
Hi,

I am trying to spark submit with G1GC for garbage collection, but it isn't working.

What is the way to deploy a spark job with G1GC?

Tried -

    spark-submit --master spark://192.168.60.20:7077 --conf -XX:+UseG1GC /appdata/bblite-codebase/test.py

Didn't work.

Tried -

    spark-submit --master spark://192.168.60.20:7077 -XX:+UseG1GC /appdata/bblite-codebase/test.py

Still didn't work. Would appreciate a help on this.

Thanks,
Aakash.
Reply | Threaded
Open this post in threaded view
|

Re: Using G1GC in Spark

srinath
You'll have to use "spark.executor.extraJavaOptions" configuration parameter:

--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC"
Regards,
Srinath.


On Thu, Jun 14, 2018 at 4:44 PM Aakash Basu <[hidden email]> wrote:
Hi,

I am trying to spark submit with G1GC for garbage collection, but it isn't working.

What is the way to deploy a spark job with G1GC?

Tried -

    spark-submit --master spark://192.168.60.20:7077 --conf -XX:+UseG1GC /appdata/bblite-codebase/test.py

Didn't work.

Tried -

    spark-submit --master spark://192.168.60.20:7077 -XX:+UseG1GC /appdata/bblite-codebase/test.py

Still didn't work. Would appreciate a help on this.

Thanks,
Aakash.
Reply | Threaded
Open this post in threaded view
|

Re: Using G1GC in Spark

Aakash Basu-2
Thanks a lot Srinath, for your perpetual help.

On Thu, Jun 14, 2018 at 5:49 PM, Srinath C <[hidden email]> wrote:
You'll have to use "spark.executor.extraJavaOptions" configuration parameter:

--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC"
Regards,
Srinath.


On Thu, Jun 14, 2018 at 4:44 PM Aakash Basu <[hidden email]> wrote:
Hi,

I am trying to spark submit with G1GC for garbage collection, but it isn't working.

What is the way to deploy a spark job with G1GC?

Tried -

    spark-submit --master spark://192.168.60.20:7077 --conf -XX:+UseG1GC /appdata/bblite-codebase/test.py

Didn't work.

Tried -

    spark-submit --master spark://192.168.60.20:7077 -XX:+UseG1GC /appdata/bblite-codebase/test.py

Still didn't work. Would appreciate a help on this.

Thanks,
Aakash.