hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4885) Update verifyBlockPlacement() API in BlockPlacementPolicy
Date Fri, 18 Oct 2013 22:05:43 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799572#comment-13799572

Tsz Wo (Nicholas), SZE commented on HDFS-4885:

- In BlockPlacementStatusDefault, "misReplicatedNum" may be misleading since it is the number
of racks the block should be additionally replicated to.  Why don't we store both requiredRacks
and currentRacks?

- hasMisplacement and getMisreplacementDescription may also be misleading since there may
not be any misplacement at all but it does not satisfy the block placement policy.  How about
renaming hasMisplacement to "isGoodPlacement" or "isPlacementPolicySatisfied", and renaming
getMisreplacementDescription to "getErrorDescription" or "getErrorMessage".

- In BlockPlacementPolicyDefault.verifyBlockPlacement(..),
  let X = Math.min(2, numberOfReplicas).  Then, X <= 2.
  We also have numRacks >= 2 since if(numRacks <= 1) return.
  As a result, X <= numRacks (i.e. Math.min(X, numRacks) == X) for any case. 
  So we could simplify the code to
int minRacks = Math.min(2, numberOfReplicas);

- Please remove the HdfsFileStatus imports in BlockPlacementPolicy and BlockPlacementPolicyDefault.

- In NamenodeFsck, let's use targetFileReplication for file.getReplication().

> Update verifyBlockPlacement() API in BlockPlacementPolicy
> ---------------------------------------------------------
>                 Key: HDFS-4885
>                 URL: https://issues.apache.org/jira/browse/HDFS-4885
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Junping Du
>            Assignee: Junping Du
>              Labels: BlockPlacementPolicy
>         Attachments: HDFS-4885.patch, HDFS-4885-v2.patch, HDFS-4885-v3.patch, HDFS-4885-v4.patch
> verifyBlockPlacement() has unused parameter -srcPath as its responsibility just verify
single block rather than files under a specific path. Also the return value (int) does not
make sense as the violation of block placement has other case than number of racks, so boolean
value should be better.

This message was sent by Atlassian JIRA

View raw message