Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 28 Oct 2011 19:01:34 +0000 (UTC)
From: "Steve Loughran (Commented) (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <1989899391.32939.1319828494490.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <534697404.5304.1319298572146.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-2492) BlockManager cross-rack replication
 checks only work for ScriptBasedMapping
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138647#comment-13138647 ] 

Steve Loughran commented on HDFS-2492:
--------------------------------------

HADOOP-7777 contains all the hadoop-common code to add a new predicate to the switch mappers (via a base class and a query method that handles mappers that are not subclasses of that); once that is in the change to block manager will be trivial.

reviews of that patch would be welcome
                
> BlockManager cross-rack replication checks only work for ScriptBasedMapping
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-2492
>                 URL: https://issues.apache.org/jira/browse/HDFS-2492
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Steve Loughran
>            Priority: Blocker
>
> The BlockManager cross-rack replication checks only works if script files are used for replication, not if alternate plugins provide the topology information.
> This is because the BlockManager sets its rack checking flag if there is a filename key
> {code}
> shouldCheckForEnoughRacks = conf.get(DFSConfigKeys.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY) != null;
> {code}
> yet this filename key is only used if the topology mapper defined by 
> {code}
> DFSConfigKeys.NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY
> {code}
> is an instance of {{ScriptBasedMapping}}
> If any other mapper is used, the system may be multi rack, but the Block Manager will not be aware of this fact unless the filename key is set to something non-null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira