hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4885) Update verifyBlockPlacement() API in BlockPlacementPolicy
Date Wed, 02 Oct 2013 00:39:24 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13783528#comment-13783528

Junping Du commented on HDFS-4885:

Thanks Luke for review and great comments!
bq. I'd use something like hasMisplacement to check misplacement in fsck instead of getMisReplicatedNum,
as the latter might not be easy calculate for certain BBPs. BlockPlacementStatus#getDescription
is probably better than either toString or getMisReplicatedString.
Sure. will update it.
bq. The advantage of this is that if we have (extended) attributes to implement various placement
grouping policies in the future, no API change is necessary.
Agree. Previous BlockPlacementConstraints is hard to extend to align with different BPP, we
should leave this flexibility to implementation of verifyBlockPlacement() in different BPP
but provide necessary info like HdfsFileStatus.
bq. We should probably add a new API to calculate/reset the accumulative status of a BBP instance,
so a proper summary can be printed. This is also useful to verify group (anti-)affinity, which
cannot be determined on a block by block basis.
That's a great idea. However, I guess it may be better to wait to implement this until the
BPP that based on (anti-) affinity group of blocks is ready, then we can see how to extend
BlockPlacementStatus (mostly seems to be merge work) to status of a bunch of blocks. Thoughts?
Will address these comments soon.

> Update verifyBlockPlacement() API in BlockPlacementPolicy
> ---------------------------------------------------------
>                 Key: HDFS-4885
>                 URL: https://issues.apache.org/jira/browse/HDFS-4885
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Junping Du
>            Assignee: Junping Du
>              Labels: BlockPlacementPolicy
>         Attachments: HDFS-4885.patch, HDFS-4885-v2.patch
> verifyBlockPlacement() has unused parameter -srcPath as its responsibility just verify
single block rather than files under a specific path. Also the return value (int) does not
make sense as the violation of block placement has other case than number of racks, so boolean
value should be better.

This message was sent by Atlassian JIRA

View raw message