Purpose of the HTTP Server?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Purpose of the HTTP Server?

Heiko Braun


Can someone briefly explain the the purpose of the HTTP server in spark? Is it related to shipping the jars around in a cluster?

Regards, Heiko
Reply | Threaded
Open this post in threaded view
|

Re: Purpose of the HTTP Server?

Cheng Lian-3
It's used for the Web UI.  By default, you may access http://localhost:8080 to view the cluster information as well as individual job details.


On Mon, Jan 27, 2014 at 10:15 PM, Heiko Braun <[hidden email]> wrote:


Can someone briefly explain the the purpose of the HTTP server in spark? Is it related to shipping the jars around in a cluster?

Regards, Heiko

Reply | Threaded
Open this post in threaded view
|

Re: Purpose of the HTTP Server?

Heiko Braun
Yes, I've seen the one used for the UI. But there is also the HttpServer and HttpFileServer. Those are the ones I was wondering about.

/Heiko

On 27 Jan 2014, at 15:18, Cheng Lian <[hidden email]> wrote:

It's used for the Web UI.  By default, you may access http://localhost:8080 to view the cluster information as well as individual job details.


On Mon, Jan 27, 2014 at 10:15 PM, Heiko Braun <[hidden email]> wrote:


Can someone briefly explain the the purpose of the HTTP server in spark? Is it related to shipping the jars around in a cluster?

Regards, Heiko


Reply | Threaded
Open this post in threaded view
|

Re: Purpose of the HTTP Server?

Mark Hamstra
Used for broadcast variables and to distribute files or jars to worker nodes.  See HttpBroadcast.scala and SparkContext.scala


On Mon, Jan 27, 2014 at 7:03 AM, Heiko Braun <[hidden email]> wrote:
Yes, I've seen the one used for the UI. But there is also the HttpServer and HttpFileServer. Those are the ones I was wondering about.

/Heiko

On 27 Jan 2014, at 15:18, Cheng Lian <[hidden email]> wrote:

It's used for the Web UI.  By default, you may access http://localhost:8080 to view the cluster information as well as individual job details.


On Mon, Jan 27, 2014 at 10:15 PM, Heiko Braun <[hidden email]> wrote:


Can someone briefly explain the the purpose of the HTTP server in spark? Is it related to shipping the jars around in a cluster?

Regards, Heiko