Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 1 Oct 2013 22:08:24 +0000 (UTC)
From: "Luke Lu (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12651354.1370531658245.14030.1380665304355@arcas>
In-Reply-To: <JIRA.12651354.1370531658245@arcas>
References: <JIRA.12651354.1370531658245@arcas>
Subject: [jira] [Commented] (HDFS-4885) Update verifyBlockPlacement() API in
 BlockPlacementPolicy
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13783405#comment-13783405 ] 

Luke Lu commented on HDFS-4885:
-------------------------------

I posted the comment before refreshing to see the v2 patch, which is getting closer. I'd use something like {{hasMisplacement}} to check misplacement in fsck instead of {{getMisReplicatedNum}}, as the latter might not be easy calculate for certain BBPs. {{BlockPlacementStatus#getDescription}} is probably better than either {{toString}} or {{getMisReplicatedString}}.

One the second thought, I wonder if we should simply pass down the {{HdfsFileStatus}} instead of just a string to {{verifyBlockPlacement}} and get rid of the constraints creation in fsck, as the file status will contain the needed info to construct/enforce proper constraints in the BPP impls.  So the API would be:
{code}
public BlockPlacementStatus verifyBlockPlacement(String srcParent, HdfsFileStatus srcFile, LocatedBlock blk);
{code}

The advantage of this is that if we have (extended) attributes to implement various placement grouping policies in the future, no API change is necessary.

We should probably add a new API to calculate/reset the accumulative status of a BBP instance, so a proper summary can be printed. This is also useful to verify group (anti-)affinity, which cannot be determined on a block by block basis.
{code}
public void resetStatus();
public BlockPlacementStatus getStatus();
{code}

Thoughts?

> Update verifyBlockPlacement() API in BlockPlacementPolicy
> ---------------------------------------------------------
>
>                 Key: HDFS-4885
>                 URL: https://issues.apache.org/jira/browse/HDFS-4885
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Junping Du
>            Assignee: Junping Du
>              Labels: BlockPlacementPolicy
>         Attachments: HDFS-4885.patch, HDFS-4885-v2.patch
>
>
> verifyBlockPlacement() has unused parameter -srcPath as its responsibility just verify single block rather than files under a specific path. Also the return value (int) does not make sense as the violation of block placement has other case than number of racks, so boolean value should be better.


--
This message was sent by Atlassian JIRA
(v6.1#6144)