hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-75) FSDataset.invalidate() throws IOException trying to delete a block which is not in the data-node blockMap
Date Tue, 31 Jan 2012 05:29:10 GMT

     [ https://issues.apache.org/jira/browse/HDFS-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tsz Wo (Nicholas), SZE resolved HDFS-75.
----------------------------------------

    Resolution: Not A Problem

It does not throw IOException now.  This is no longer a problem.
                
> FSDataset.invalidate() throws IOException trying to delete a block which is not in the
data-node blockMap
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-75
>                 URL: https://issues.apache.org/jira/browse/HDFS-75
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Konstantin Shvachko
>
> Here is what I get in the data-node log. This is happening regularly, but not all the
time.
> {code}
> 07/08/21 15:22:20 INFO dfs.DataNode: Deleting block blk_5351209297984813226 file /hadoop-data/ndfs/data/current/blk_5351209297984813226
> 07/08/21 15:22:20 INFO dfs.DataNode: Deleting block blk_8240763237624393278 file /hadoop-data/ndfs/data/current/blk_8240763237624393278
> 07/08/21 15:22:21 WARN dfs.DataNode: Unexpected error trying to delete block blk_5351209297984813226.
Block not found in blockMap. 
> 07/08/21 15:22:21 WARN dfs.DataNode: Unexpected error trying to delete block blk_8240763237624393278.
Block not found in blockMap. 
> 07/08/21 15:22:21 WARN dfs.DataNode: java.io.IOException: Error in deleting blocks.
> 	at org.apache.hadoop.dfs.FSDataset.invalidate(FSDataset.java:714)
> 	at org.apache.hadoop.dfs.DataNode.processCommand(DataNode.java:590)
> 	at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:493)
> 	at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1310)
> 	at java.lang.Thread.run(Thread.java:595)
> {code}
> The data-node successfully removed two blocks and then one second later received another
request 
> from the name-node to remove the same two blocks.
> # I don't  see why the data-node should throw an exception in this case at all. It is
a clear warning.
>    Even if it does it should use the same description as in the warning message.
> # Name-node calculates blocks to remove for a data-node in two cases: a heartbeat and
a block-report.
>    I suspect that this is what happened here. The 2 blocks were first requested to be
removed in reply to
>    the heartbeat and the in reply to the block report (or vice versa). 
>    The question is why name-node returns the same blocks twice?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message