hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8418) Fix the isNeededReplication calculation for Striped block in NN
Date Mon, 18 May 2015 05:23:00 GMT
Yi Liu created HDFS-8418:

             Summary: Fix the isNeededReplication calculation for Striped block in NN
                 Key: HDFS-8418
                 URL: https://issues.apache.org/jira/browse/HDFS-8418
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Yi Liu
            Assignee: Yi Liu
            Priority: Critical

Currently when calculating {{isNeededReplication}} for striped block, we use BlockCollection#getPreferredBlockReplication
to get expected replica number for striped block. See an example:
public void checkReplication(BlockCollection bc) {
    final short expected = bc.getPreferredBlockReplication();
    for (BlockInfo block : bc.getBlocks()) {
      final NumberReplicas n = countNodes(block);
      if (isNeededReplication(block, expected, n.liveReplicas())) { 
        neededReplications.add(block, n.liveReplicas(),
            n.decommissionedAndDecommissioning(), expected);
      } else if (n.liveReplicas() > expected) {
        processOverReplicatedBlock(block, expected, null, null);
But actually it's not correct, for example, if the length of striped file is less than a cell,
then the expected replica of the block should be {{1 + parityBlkNum}}. 

This message was sent by Atlassian JIRA

View raw message