hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6830) BlockManager.addStorage fails when DN updates storage
Date Fri, 08 Aug 2014 06:42:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14090388#comment-14090388

Arpit Agarwal commented on HDFS-6830:

Hi Nicholas, thanks for looking at the change. By callee I meant BlockInfo.removeStorage().
removeStorage() expects that the block is not in the blocklist of any DatanodeStorageInfo
i.e. the removal from the blocklist has been done prior to removing the backlink from BlockInfo
to DatanodeStorage.

DatanodeStorageInfo.addBlock was not doing the removal from otherStorage's list. The fix is
to do so now. You can see this assert fire if you revert the code changes from my patch except
the test case. Then the following assert fires.

  boolean removeStorage(DatanodeStorageInfo storage) {
    int dnIndex = findStorageInfo(storage);
    if(dnIndex < 0) // the node is not found
      return false;
    assert getPrevious(dnIndex) == null && getNext(dnIndex) == null : 
      "Block is still in the list and must be removed first.";

> BlockManager.addStorage fails when DN updates storage
> -----------------------------------------------------
>                 Key: HDFS-6830
>                 URL: https://issues.apache.org/jira/browse/HDFS-6830
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.5.0
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>         Attachments: HDFS-6830.01.patch, HDFS-6830.02.patch, HDFS-6830.03.patch
> The call to {{removeStorageInfo}} is wrong because the block is still in the DatanodeStorage's
list of blocks and the callee does not expect it to be.
> {code}
>       } else {
>         // The block is on the DN but belongs to a different storage.
>         // Update our state.
>         removeStorage(getStorageInfo(idx));
>         added = false;      // Just updating storage. Return false.
>       }
> {code}
> It is a very unlikely code path to hit since storage updates usually occur via incremental
block reports.

This message was sent by Atlassian JIRA

View raw message