hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-10858) FBR processing may generate incorrect reportedBlock-blockGroup mapping
Date Mon, 12 Sep 2016 18:22:20 GMT
Jing Zhao created HDFS-10858:

             Summary: FBR processing may generate incorrect reportedBlock-blockGroup mapping
                 Key: HDFS-10858
                 URL: https://issues.apache.org/jira/browse/HDFS-10858
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: erasure-coding
    Affects Versions: 3.0.0-alpha1
            Reporter: Jing Zhao
            Assignee: Jing Zhao
            Priority: Blocker

In BlockManager#reportDiffSorted:
    } else if (reportedState == ReplicaState.FINALIZED &&
               (storedBlock.findStorageInfo(storageInfo) == -1 ||
                corruptReplicas.isReplicaCorrupt(storedBlock, dn))) {
      // Add replica if appropriate. If the replica was previously corrupt
      // but now okay, it might need to be updated.
      toAdd.add(new BlockInfoToAdd(storedBlock, replica));

"new BlockInfoToAdd(storedBlock, replica)" is wrong because "replica" (i.e., the reported
block) is a reused object provided by BlockListAsLongs#iterator. Later this object is reused
by directly changing its ID/GS. Thus {{addStoredBlock}} can get wrong (reportedBlock, stored-BlockInfo)
mapping. For EC the reported block is used to calculate the internal block index. Thus the
bug can completely corrupt the EC block group internal states.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message