hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4885) Update verifyBlockPlacement() API in BlockPlacementPolicy
Date Tue, 01 Oct 2013 22:08:24 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13783405#comment-13783405

Luke Lu commented on HDFS-4885:

I posted the comment before refreshing to see the v2 patch, which is getting closer. I'd use
something like {{hasMisplacement}} to check misplacement in fsck instead of {{getMisReplicatedNum}},
as the latter might not be easy calculate for certain BBPs. {{BlockPlacementStatus#getDescription}}
is probably better than either {{toString}} or {{getMisReplicatedString}}.

One the second thought, I wonder if we should simply pass down the {{HdfsFileStatus}} instead
of just a string to {{verifyBlockPlacement}} and get rid of the constraints creation in fsck,
as the file status will contain the needed info to construct/enforce proper constraints in
the BPP impls.  So the API would be:
public BlockPlacementStatus verifyBlockPlacement(String srcParent, HdfsFileStatus srcFile,
LocatedBlock blk);

The advantage of this is that if we have (extended) attributes to implement various placement
grouping policies in the future, no API change is necessary.

We should probably add a new API to calculate/reset the accumulative status of a BBP instance,
so a proper summary can be printed. This is also useful to verify group (anti-)affinity, which
cannot be determined on a block by block basis.
public void resetStatus();
public BlockPlacementStatus getStatus();


> Update verifyBlockPlacement() API in BlockPlacementPolicy
> ---------------------------------------------------------
>                 Key: HDFS-4885
>                 URL: https://issues.apache.org/jira/browse/HDFS-4885
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Junping Du
>            Assignee: Junping Du
>              Labels: BlockPlacementPolicy
>         Attachments: HDFS-4885.patch, HDFS-4885-v2.patch
> verifyBlockPlacement() has unused parameter -srcPath as its responsibility just verify
single block rather than files under a specific path. Also the return value (int) does not
make sense as the violation of block placement has other case than number of racks, so boolean
value should be better.

This message was sent by Atlassian JIRA

View raw message