hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4885) Update verifyBlockPlacement() API in BlockPlacementPolicy
Date Tue, 01 Oct 2013 22:08:24 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13783405#comment-13783405
] 

Luke Lu commented on HDFS-4885:
-------------------------------

I posted the comment before refreshing to see the v2 patch, which is getting closer. I'd use
something like {{hasMisplacement}} to check misplacement in fsck instead of {{getMisReplicatedNum}},
as the latter might not be easy calculate for certain BBPs. {{BlockPlacementStatus#getDescription}}
is probably better than either {{toString}} or {{getMisReplicatedString}}.

One the second thought, I wonder if we should simply pass down the {{HdfsFileStatus}} instead
of just a string to {{verifyBlockPlacement}} and get rid of the constraints creation in fsck,
as the file status will contain the needed info to construct/enforce proper constraints in
the BPP impls.  So the API would be:
{code}
public BlockPlacementStatus verifyBlockPlacement(String srcParent, HdfsFileStatus srcFile,
LocatedBlock blk);
{code}

The advantage of this is that if we have (extended) attributes to implement various placement
grouping policies in the future, no API change is necessary.

We should probably add a new API to calculate/reset the accumulative status of a BBP instance,
so a proper summary can be printed. This is also useful to verify group (anti-)affinity, which
cannot be determined on a block by block basis.
{code}
public void resetStatus();
public BlockPlacementStatus getStatus();
{code}

Thoughts?

> Update verifyBlockPlacement() API in BlockPlacementPolicy
> ---------------------------------------------------------
>
>                 Key: HDFS-4885
>                 URL: https://issues.apache.org/jira/browse/HDFS-4885
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Junping Du
>            Assignee: Junping Du
>              Labels: BlockPlacementPolicy
>         Attachments: HDFS-4885.patch, HDFS-4885-v2.patch
>
>
> verifyBlockPlacement() has unused parameter -srcPath as its responsibility just verify
single block rather than files under a specific path. Also the return value (int) does not
make sense as the violation of block placement has other case than number of racks, so boolean
value should be better.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message