hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2492) BlockManager cross-rack replication checks only work for ScriptBasedMapping
Date Sat, 29 Oct 2011 14:43:32 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steve Loughran updated HDFS-2492:
---------------------------------

    Attachment: HDFS-2492-blockmanager.patch

This is the patch to the block manager that uses HADOOP-7777 to state when a DNS mapper is
explicitly single switch. 

It adds a public getter on the DatanodeManager to provide access to its DNS mapping; this
could be downgraded to package-scope.

I'm not submitting this as a patch for automated review (yet) because of its dependencies.
                
> BlockManager cross-rack replication checks only work for ScriptBasedMapping
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-2492
>                 URL: https://issues.apache.org/jira/browse/HDFS-2492
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Steve Loughran
>            Priority: Blocker
>         Attachments: HDFS-2492-blockmanager.patch
>
>
> The BlockManager cross-rack replication checks only works if script files are used for
replication, not if alternate plugins provide the topology information.
> This is because the BlockManager sets its rack checking flag if there is a filename key
> {code}
> shouldCheckForEnoughRacks = conf.get(DFSConfigKeys.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY)
!= null;
> {code}
> yet this filename key is only used if the topology mapper defined by 
> {code}
> DFSConfigKeys.NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY
> {code}
> is an instance of {{ScriptBasedMapping}}
> If any other mapper is used, the system may be multi rack, but the Block Manager will
not be aware of this fact unless the filename key is set to something non-null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message