hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-140) When a file is deleted, its blocks remain in the blocksmap till the next block report from Datanode
Date Wed, 12 Oct 2011 08:33:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125663#comment-13125663
] 

Uma Maheswara Rao G commented on HDFS-140:
------------------------------------------

Hi Todd,

 Thanks a lot for taking a look!

I have verified this in trunk codebase. this issue has been addressed already.

{code}
 public void removeBlock(Block block) {
    block.setNumBytes(BlockCommand.NO_ACK);
    addToInvalidates(block);
    corruptReplicas.removeFromCorruptReplicasMap(block);
    blocksMap.removeBlock(block);
  }
{code}

FsNameSystem#removePathAndBlock will invoke this API. This is making sure that, block has
been removed from blocksMap.
It looks to me that 0.21 onwards, this issue has been fixed. But i did not get exact JIRA
specific to this problem.

But I am using 20.2 version. When i profiled, i found that blockMap elements are growing.
Observation is that, after blockreports, that has been come down.

So,By looking at the code, current versions in 20X( *20.205,20.206* ) also has this problem.

Thanks
Uma


                
> When a file is deleted, its blocks remain in the blocksmap till the next block report
from Datanode
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-140
>                 URL: https://issues.apache.org/jira/browse/HDFS-140
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.20.1
>            Reporter: dhruba borthakur
>            Assignee: Uma Maheswara Rao G
>             Fix For: 0.20.205.0
>
>         Attachments: HDFS-140.20security205.patch
>
>
> When a file is deleted, the namenode sends out block deletions messages to the appropriate
datanodes. However, the namenode does not delete these blocks from the blocksmap. Instead,
the processing of the next block report from the datanode causes these blocks to get removed
from the blocksmap.
> If we desire to make block report processing less frequent, this issue needs to be addressed.
Also, this introduces indeterministic behaviout to a a few unit tests. Another factor to consider
is to ensure that duplicate block detection is not compromised.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message