hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8418) Fix the isNeededReplication calculation for Striped block in NN
Date Wed, 30 Sep 2015 16:39:44 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937673#comment-14937673
] 

Hudson commented on HDFS-8418:
------------------------------

FAILURE: Integrated in Hadoop-Yarn-trunk #1203 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1203/])
HDFS-8418. Fix the isNeededReplication calculation for Striped block in (zhezhang: rev b008348dbf9bdd5070930be5d182116c5d370f6b)
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt


> Fix the isNeededReplication calculation for Striped block in NN
> ---------------------------------------------------------------
>
>                 Key: HDFS-8418
>                 URL: https://issues.apache.org/jira/browse/HDFS-8418
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>            Priority: Critical
>             Fix For: HDFS-7285
>
>         Attachments: HDFS-8418-HDFS-7285.001.patch
>
>
> Currently when calculating {{isNeededReplication}} for striped block, we use BlockCollection#getPreferredBlockReplication
to get expected replica number for striped block. See an example:
> {code}
> public void checkReplication(BlockCollection bc) {
>     final short expected = bc.getPreferredBlockReplication();
>     for (BlockInfo block : bc.getBlocks()) {
>       final NumberReplicas n = countNodes(block);
>       if (isNeededReplication(block, expected, n.liveReplicas())) { 
>         neededReplications.add(block, n.liveReplicas(),
>             n.decommissionedAndDecommissioning(), expected);
>       } else if (n.liveReplicas() > expected) {
>         processOverReplicatedBlock(block, expected, null, null);
>       }
>     }
>   }
> {code}
> But actually it's not correct, for example, if the length of striped file is less than
a cell, then the expected replica of the block should be {{1 + parityBlkNum}} instead of {{dataBlkNum
+ parityBlkNum}}. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message