How to stop a running SparkContext in the proper way?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to stop a running SparkContext in the proper way?

MEETHU MATHEW
Hi,

I want to know how I can stop a running SparkContext in a proper way so that next time when I start a new SparkContext, the web UI can be launched on the same port 4040.Now when i quit the job using ctrl+z the new sc are launched in new ports.

I have the same problem with ipython notebook.It is launched on a different port when I start the notebook second time after closing the first one.I am starting ipython using the command

IPYTHON_OPTS="notebook --ip xxxx --pylab inline" ./bin/pyspark

Thanks & Regards,
Meethu M
Reply | Threaded
Open this post in threaded view
|

Re: How to stop a running SparkContext in the proper way?

Xiangrui Meng
Did you try sc.stop()?

On Tue, Jun 3, 2014 at 9:54 PM, MEETHU MATHEW <[hidden email]> wrote:

> Hi,
>
> I want to know how I can stop a running SparkContext in a proper way so that
> next time when I start a new SparkContext, the web UI can be launched on the
> same port 4040.Now when i quit the job using ctrl+z the new sc are launched
> in new ports.
>
> I have the same problem with ipython notebook.It is launched on a different
> port when I start the notebook second time after closing the first one.I am
> starting ipython using the command
>
> IPYTHON_OPTS="notebook --ip xxxx --pylab inline" ./bin/pyspark
>
> Thanks & Regards,
> Meethu M
Reply | Threaded
Open this post in threaded view
|

Re: How to stop a running SparkContext in the proper way?

Akhil
In reply to this post by MEETHU MATHEW
ctrl + z will stop the job from being executed ( If you do a fg/bg you can resume the job). You need to press ctrl + c to terminate the job!

Thanks
Best Regards


On Wed, Jun 4, 2014 at 10:24 AM, MEETHU MATHEW <[hidden email]> wrote:
Hi,

I want to know how I can stop a running SparkContext in a proper way so that next time when I start a new SparkContext, the web UI can be launched on the same port 4040.Now when i quit the job using ctrl+z the new sc are launched in new ports.

I have the same problem with ipython notebook.It is launched on a different port when I start the notebook second time after closing the first one.I am starting ipython using the command

IPYTHON_OPTS="notebook --ip xxxx --pylab inline" ./bin/pyspark

Thanks & Regards,
Meethu M