Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Date: Sat, 18 Feb 2012 13:44:59 +0000 (UTC)
From: "Steve Loughran (Created) (JIRA)" <jira@apache.org>
To: common-dev@hadoop.apache.org
Message-ID: 
 <613333825.54154.1329572699445.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Created] (HADOOP-8091) Add a topology mapper that reads
 hostname to rack mappings from a Java properties file
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Add a topology mapper that reads hostname to rack mappings from a Java properties file
--------------------------------------------------------------------------------------

                 Key: HADOOP-8091
                 URL: https://issues.apache.org/jira/browse/HADOOP-8091
             Project: Hadoop Common
          Issue Type: New Feature
          Components: util
    Affects Versions: 0.24.0
            Reporter: Steve Loughran
            Priority: Minor


Getting mapping scripts right is surprisingly hard -and if its wrong in production bad things happen. It would be good to have something simpler for beginners -and one that is trivial to generate by a machine based on infrastructure data.

I propose adding an alternative mapper, one driven by a java property file

 # the specific topology mapper must be identified for loading
 # it uses another key to identify the property file to load. This is checked for on startup -if missing, fail.
 # one property, perhaps "default-rack" identifies the default rack mapping for any host not in the list
 # every other entry lists a hostname to rack mapping
 # hostname mapping is done on the first entry in the FQDN, to be less brittle to domain resolution.

Example
{code}
default-rack=/rack1
host1=/rack1
host2=/rack1
host3=/rack2
host4=/rack2
{code}

Implementation
 * add a new mapper that builds a concurrent hash map
 * read in every entry in the specified property file, add it to the map
 * when queried, extract the hostname (i.e. everything before any ".")
 * match that in the hash table, return if found
 * if not found: return the default rack

Feature creep would be to poll this file for changes at a (specified) frequency, and pick up the changes when they occur. This would require removing the caching topology mapper that wraps all others in the NN and RM.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira