Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 16 Nov 2012 18:12:12 +0000 (UTC)
From: "Jing Zhao (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <1612443476.124967.1353089532774.JavaMail.jiratomcat@arcas>
In-Reply-To: <740418511.85756.1347822428737.JavaMail.jiratomcat@arcas>
Subject: [jira] [Updated] (HDFS-3941) Backport HDFS-3498 and HDFS3601:
 update replica placement policy for new added "NodeGroup" layer topology
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jing Zhao updated HDFS-3941:
----------------------------

    Attachment: HDFS-3941.003.patch

Junping, thanks for the comments! You're right, my 002 patch updates numOfAvailableNodes. The new 003 patch removes this part, also remove part of the testcases from TestReplicationPolicyWithNodeGroup which are not included in the trunk currently.
                
> Backport HDFS-3498 and HDFS3601: update replica placement policy for new added "NodeGroup" layer topology
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3941
>                 URL: https://issues.apache.org/jira/browse/HDFS-3941
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: name-node
>    Affects Versions: 1.0.0
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: HDFS-3941.002.patch, HDFS-3941.003.patch, HDFS-3941.patch
>
>
> With enabling additional layer of "NodeGroup", the replica placement policy used in BlockPlacementPolicyWithNodeGroup is updated to following rules:
> 0. No more than one replica is placed within a NodeGroup (*)
> 1. First replica on the local node.
> 2. Second and third replicas are within the same rack but remote rack with 1st replica.
> 3. Other replicas on random nodes with restriction that no more than two replicas are placed in the same rack, if there is enough racks.
> Also, this patch abstract Replica Removal Policy from FSNameSystem to BlockPlacementPolicy and update removal policy slightly to remove duplicated replica within the same NodeGroup first when over-replicated happens. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira