hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4168) Support multiple network interfaces
Date Tue, 30 Dec 2014 17:26:15 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14261268#comment-14261268

Steve Loughran commented on MAPREDUCE-4168:

bq. The biggest problem that we have in Hadoop is that we use the server-side network configuration
as the same information we pass to or expect clients to use for connection information. In
multi-home situations, these may not be (and usually aren't) the same. So if a service is
configured to use (to bind to all interfaces), the client can't use that to connect
to it since that doesn't resolve to anything usable.

The issue here is the expectation that clients will magically get valid -site.xml files, where
the definition of valid is : valid at time {{t}} for client {{c}} to submit jobs to cluster
{{h}}. The configurations used within cluster {{h}} may not be valid for client {{c}} even
for the same time value {{t}}. This is why, although you can point your browser at http://resourcemanager:8090/config,
you have no guarantees of getting a usable config.

Allen, if you look closely at YARN-896 you'll see that the registry can live standalone, and
has special space reserved {{/services}} for us to publish service records for system-wide
services, service records that could include the URL to an API which lets you enum configs
and retrieve them. Which is precisely what we offer in Slider for pushing out dynamically
generated hbase-site.xml, accumulo-site.xml, etc. 

The fact that there is effectively a placeholder for publishing core-site XML &c is not
a coincidence. It's something we need.

> Support multiple network interfaces
> -----------------------------------
>                 Key: MAPREDUCE-4168
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4168
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Tom White
> Umbrella jira to track the MapReduce side of HADOOP-8198.

This message was sent by Atlassian JIRA

View raw message