Problem with master webui with reverse proxy when workers >= 10

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Problem with master webui with reverse proxy when workers >= 10

tmckay
This post has NOT been accepted by the mailing list yet.
Hi folks,

  I'm running a containerized spark 2.1.0 setup.  If I have reverse proxy turned on, everything is fine if I have fewer than 10 workers.  If I create a cluster with 10 or more, the master web ui is unavailable.

This is even true for the same cluster if I start with a lower number and add workers to bring the total above 10.  The web ui is available until I cross that boundary, then becomes unavailable.

If I run without reverse proxy, I can create clusters of 10+ (20, 30, 40 ...) and the console is always available.

Any ideas? Has anyone seen this behavior on bear metal, or VMs, or with containers?
Any logging I can turn on in the master webserver that might shed some light?

Thanks,

Trevor McKay