hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hui Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7994) Detect if resevered EC Block ID is already used
Date Thu, 09 Apr 2015 11:20:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14487162#comment-14487162

Hui Zheng commented on HDFS-7994:

Hi [~szetszwo]
I want to confirm my understanding about this issue.Is it correct?
When the NameNode loads a fsimage,it doesn't care about whether the block is an EC Block during
putting the blocks into the BlocksMap.
But it should take different lookup actions for non-EC/EC blocks during processing BlockReport.
For  non-EC blocks it only does a lookup in the blocksmap by blockid.
For EC blocks it must convert the blockid to EC-Blockid(the blockid of the first block of
a EC block group).

I think the current implementation has complete this function.
  public BlockInfo getStoredBlock(Block block) {
    BlockInfo info = null;
    if (BlockIdManager.isStripedBlockID(block.getBlockId())) {
      info = blocksMap.getStoredBlock(
          new Block(BlockIdManager.convertToStripedID(block.getBlockId())));     //AAA
    if (info == null) {
      info = blocksMap.getStoredBlock(block);              // BBB
    return info;
1. A StripedBlockId(whether or not the low 4 bits are not '0000') comes and the block is really
EC block, it can be found by AAA.
2. A StripedBlockId whose low 4 bits are '0000' comes but the block is not EC, it can also
be found by AAA.
3. A StripedBlockId whose low 4 bits are NOT '0000' comes but the block is not EC, it can
be found by BBB.
3. A non-StripedBlockId comes,it can be found by BBB.

So I don't think we need to process this issue.

> Detect if resevered EC Block ID is already used
> -----------------------------------------------
>                 Key: HDFS-7994
>                 URL: https://issues.apache.org/jira/browse/HDFS-7994
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Hui Zheng
> Since random block IDs were supported by some early version of HDFS, the block ID reserved
for EC blocks could be already used by some existing blocks in a cluster. During NameNode
startup, it detects if there are reserved EC block IDs used by non-EC blocks. If it is the
case, NameNode will do an additional blocksMap lookup when there is a miss in a blockGroupsMap

This message was sent by Atlassian JIRA

View raw message