flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shannon Carey <sca...@expedia.com>
Subject JobManager HA behind load balancer
Date Wed, 16 Aug 2017 15:24:10 GMT
Is anyone running multiple JobManagers (in High Availability mode) behind a load balancer such
as an AWS ELB or a software proxy such as HAProxy or Nginx?

Right now, it appears that server-side redirects that come from the JobManager Web UI use
the internal IP address of the JobManager (from Akka). Therefore, if you're accessing your
JobManager via a DNS name or a load balancing proxy, the redirect doesn't work properly. Has
anyone created a workaround for this?

If there's no workaround, should we perhaps add a config setting to the JobManager to tell
it what DNS name or root URL to use when sending redirect responses?

Also, it looks like at least some types of requests are not supported by non-master JobManagers,
and therefore they respond with a redirect to the internal address of the master. Is it necessary
to integrate the proxy with Zookeeper so that requests will only be proxied to the master
JobManager? If the non-master nodes only send redirects, then including them as upstream servers
in the proxy would be problematic.

Thanks for the info,
View raw message