hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8091) Add a topology mapper that reads hostname to rack mappings from a Java properties file
Date Sun, 19 Feb 2012 17:38:35 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211437#comment-13211437
] 

Aaron T. Myers commented on HADOOP-8091:
----------------------------------------

Seems like this might be (very nearly) a duplicate of HADOOP-7030. Do you agree, Steve?
                
> Add a topology mapper that reads hostname to rack mappings from a Java properties file
> --------------------------------------------------------------------------------------
>
>                 Key: HADOOP-8091
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8091
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: util
>    Affects Versions: 0.24.0
>            Reporter: Steve Loughran
>            Priority: Minor
>
> Getting mapping scripts right is surprisingly hard -and if its wrong in production bad
things happen. It would be good to have something simpler for beginners -and one that is trivial
to generate by a machine based on infrastructure data.
> I propose adding an alternative mapper, one driven by a java property file
>  # the specific topology mapper must be identified for loading
>  # it uses another key to identify the property file to load. This is checked for on
startup -if missing, fail.
>  # one property, perhaps "default-rack" identifies the default rack mapping for any host
not in the list
>  # every other entry lists a hostname to rack mapping
>  # hostname mapping is done on the first entry in the FQDN, to be less brittle to domain
resolution.
> Example
> {code}
> default-rack=/rack1
> host1=/rack1
> host2=/rack1
> host3=/rack2
> host4=/rack2
> {code}
> Implementation
>  * add a new mapper that builds a concurrent hash map
>  * read in every entry in the specified property file, add it to the map
>  * when queried, extract the hostname (i.e. everything before any ".")
>  * match that in the hash table, return if found
>  * if not found: return the default rack
> Feature creep would be to poll this file for changes at a (specified) frequency, and
pick up the changes when they occur. This would require removing the caching topology mapper
that wraps all others in the NN and RM.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message