flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shannon Carey <sca...@expedia.com>
Subject Flink YARN job manager web port
Date Thu, 21 Apr 2016 20:55:35 GMT
The documentation states: "The ports Flink is using for its services are the standard ports
configured by the user + the application id as an offset"

When I launch Flink via YARN in an AWS EMR cluster, stdout says:
JobManager Web Interface: http://ip-xxx.us-west-2.compute.internal:20888/proxy/application_1461178294210_0010/

I need to be able to create an IAM Security Group that allows access to the JobManager web
interface so that I can make use of it. However, I am confused about how port 20888 is chosen.
Based on the code, I would have guessed that it would use the same port as given by: "yarn
application -status application_1461178294210_0010". However, that's not the case (they don't
match). It gives "Tracking-URL : http://ip-xxx.us-west-2.compute.internal:36495"

On the other hand, I see that YarnApplicationMasterRunner sets the port to 0, which InetSocketAddress
says results in "A port number of zero will let the system pick up an ephemeral port in a
bind operation."

I couldn't find anything in the code that adds an offset to a port. Changing the value of
"jobmanager.web.port" appears to have no effect. The documentation on "Running Flink on YARN
behind Firewalls" only talks about the JobManager and BlobServer ports.

Does Flink need logic to allow users to specify a range of ports for jobmanager.web.port in
the same way as is done in BootstrapTools#startActorSystem? If so, I am happy to make that
contribution!

-Shannon
Mime
View raw message