hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-3941) Backport HDFS-3498 and HDFS3601: update replica placement policy for new added "NodeGroup" layer topology
Date Sat, 17 Nov 2012 02:55:12 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jing Zhao updated HDFS-3941:
----------------------------

    Attachment: HDFS-3941.005.patch

Updated the patch to make it more compatible with the trunk. Have run unit tests locally and
all the testcases passed except TestNNThroughputBenchmark (reported in HDFS-4204).
                
> Backport HDFS-3498 and HDFS3601: update replica placement policy for new added "NodeGroup"
layer topology
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3941
>                 URL: https://issues.apache.org/jira/browse/HDFS-3941
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: name-node
>    Affects Versions: 1.0.0
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: HDFS-3941.002.patch, HDFS-3941.003.patch, HDFS-3941.004.patch, HDFS-3941.005.patch,
HDFS-3941.patch
>
>
> With enabling additional layer of "NodeGroup", the replica placement policy used in BlockPlacementPolicyWithNodeGroup
is updated to following rules:
> 0. No more than one replica is placed within a NodeGroup (*)
> 1. First replica on the local node.
> 2. Second and third replicas are within the same rack but remote rack with 1st replica.
> 3. Other replicas on random nodes with restriction that no more than two replicas are
placed in the same rack, if there is enough racks.
> Also, this patch abstract Replica Removal Policy from FSNameSystem to BlockPlacementPolicy
and update removal policy slightly to remove duplicated replica within the same NodeGroup
first when over-replicated happens. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message