Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 14 Oct 2015 17:41:05 +0000 (UTC)
From: "Brahma Reddy Battula (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12839747.1435035538000.39242.1444844465676@Atlassian.JIRA>
In-Reply-To: <JIRA.12839747.1435035538000@Atlassian.JIRA>
References: <JIRA.12839747.1435035538000@Atlassian.JIRA>
 <JIRA.12839747.1435035538865@arcas>
Subject: [jira] [Commented] (HDFS-8647) Abstract BlockManager's rack policy
 into BlockPlacementPolicy
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14957361#comment-14957361 ] 

Brahma Reddy Battula commented on HDFS-8647:
--------------------------------------------

[~mingma] and [~vinayrpet]

Uploaded patch to address following comments....

{quote}The existing blockHasEnoughRacksStriped compares getRealDataBlockNum (# of data blocks) with the number racks. But after the refactoring, it compares getRealTotalBlockNum (# of total blocks) with the number racks,{quote}
{quote}
It might be easier to not to pass isStriped to verifyBlockPlacement. Instead, have BlockPlacementPolicyRackFaultTolerant implement verifyBlockPlacement which will use numberOfReplicas as the minRacks. This also makes the patch more applicable to branch-2 which doesn't have striped EC
{quote}


> Abstract BlockManager's rack policy into BlockPlacementPolicy
> -------------------------------------------------------------
>
>                 Key: HDFS-8647
>                 URL: https://issues.apache.org/jira/browse/HDFS-8647
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ming Ma
>            Assignee: Brahma Reddy Battula
>         Attachments: HDFS-8647-001.patch, HDFS-8647-002.patch, HDFS-8647-003.patch, HDFS-8647-004.patch, HDFS-8647-004.patch, HDFS-8647-005.patch, HDFS-8647-006.patch
>
>
> Sometimes we want to have namenode use alternative block placement policy such as upgrade domains in HDFS-7541.
> BlockManager has built-in assumption about rack policy in functions such as useDelHint, blockHasEnoughRacks. That means when we have new block placement policy, we need to modify BlockManager to account for the new policy. Ideally BlockManager should ask BlockPlacementPolicy object instead. That will allow us to provide new BlockPlacementPolicy without changing BlockManager.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)