hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Craig Welch (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
Date Wed, 30 Jul 2014 00:24:40 GMT

    [ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078685#comment-14078685

Craig Welch commented on YARN-1994:

Hmm, I think I understand the scenario.  As I understand it, this will only be a problem if
you want to refer to the host by a value other than the "main" hostname, e.g. something other
than what would be returned from InetAddress.getLocalHost().getHostName() - as that is what
it will come back with in the bind-host case.  (It's not that the results are indeterminate
- it returns the "primary" host name, it's that you want to use something other than the primary
host name).  There are many places in the code where this convention is used to determine
the name of the host - it's an implicit convention at least, and I'm concerned that there
will be cases where this mismatch causes issues, as evidenced by all of the various things
which needed to be tracked down/etc.  Also, this is only realistic for single address services
(or rather for cases where addresses are enumerated, including single addresses or ha resource
manager, etc), others will not have an address in a centralized configuration to "override"

Instead of this, could you use the primary hostname and control what address it uses through
name resolution?  Inside the cluster the hostname should resolve to only the infiniband addresses,
that way only those will be used (the two ib addresses).  From external networks you can set
the resolution to the ethernet address as you mentioned above.  This way, the host is always
referred to by the same name, clients are always able to get to it over the desired interface,
and no changes to the connect address logic are required.

> Expose YARN/MR endpoints on multiple interfaces
> -----------------------------------------------
>                 Key: YARN-1994
>                 URL: https://issues.apache.org/jira/browse/YARN-1994
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager, resourcemanager, webapp
>    Affects Versions: 2.4.0
>            Reporter: Arpit Agarwal
>            Assignee: Craig Welch
>         Attachments: YARN-1994.0.patch, YARN-1994.1.patch, YARN-1994.11.patch, YARN-1994.11.patch,
YARN-1994.12.patch, YARN-1994.2.patch, YARN-1994.3.patch, YARN-1994.4.patch, YARN-1994.5.patch,
YARN-1994.6.patch, YARN-1994.7.patch
> YARN and MapReduce daemons currently do not support specifying a wildcard address for
the server endpoints. This prevents the endpoints from being accessible from all interfaces
on a multihomed machine.
> Note that if we do specify INADDR_ANY for any of the options, it will break clients as
they will attempt to connect to We need a solution that allows specifying a hostname
or IP-address for clients while requesting wildcard bind for the servers.
> (List of endpoints is in a comment below)

This message was sent by Atlassian JIRA

View raw message