hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Singhi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3157) Error in deleting block is keep on coming from DN even after the block report and directory scanning has happened
Date Fri, 18 May 2012 12:50:08 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278778#comment-13278778
] 

Ashish Singhi commented on HDFS-3157:
-------------------------------------

Patch updated and ready for review.
Please provide your review comments.
{code}
+      /*
+       * Look up again to storedBlock as it might be a reported block also.
+       * @see BlockManager#checkReplicaCorrupt(...)
+       */
+      BlockInfo blkInfo = blocksMap.getStoredBlock(storedBlock);
{code}
I have added this because in updatedNeededReplication we want namenode to ask datanode to
replicate the storedBlock which is there in its blockMap not with the reported block with
datanode is reporting as corrupt.

Now in the test case I am asserting 3 things, 
First - There should be one block in the corruptReplicasMap.
Second - After marking the block as corrupt, ReplicationMonitor thread should replicate a
live replica to one of the datanode.
Third - After replicating the live replica, the corrupt replica in corruptReplicasMap should
get invalidated.
                
> Error in deleting block is keep on coming from DN even after the block report and directory
scanning has happened
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3157
>                 URL: https://issues.apache.org/jira/browse/HDFS-3157
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: J.Andreina
>            Assignee: Ashish Singhi
>             Fix For: 2.0.0, 3.0.0
>
>         Attachments: HDFS-3157-1.patch, HDFS-3157.patch, HDFS-3157.patch, HDFS-3157.patch
>
>
> Cluster setup:
> 1NN,Three DN(DN1,DN2,DN3),replication factor-2,"dfs.blockreport.intervalMsec" 300,"dfs.datanode.directoryscan.interval"
1
> step 1: write one file "a.txt" with sync(not closed)
> step 2: Delete the blocks in one of the datanode say DN1(from rbw) to which replication
happened.
> step 3: close the file.
> Since the replication factor is 2 the blocks are replicated to the other datanode.
> Then at the NN side the following cmd is issued to DN from which the block is deleted
> -------------------------------------------------------------------------------------
> {noformat}
> 2012-03-19 13:41:36,905 INFO org.apache.hadoop.hdfs.StateChange: BLOCK NameSystem.addToCorruptReplicasMap:
duplicate requested for blk_2903555284838653156 to add as corrupt on XX.XX.XX.XX by /XX.XX.XX.XX
because reported RBW replica with genstamp 1002 does not match COMPLETE block's genstamp in
block map 1003
> 2012-03-19 13:41:39,588 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* Removing block
blk_2903555284838653156_1003 from neededReplications as it has enough replicas.
> {noformat}
> From the datanode side in which the block is deleted the following exception occured
> {noformat}
> 2012-02-29 13:54:13,126 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Unexpected
error trying to delete block blk_2903555284838653156_1003. BlockInfo not found in volumeMap.
> 2012-02-29 13:54:13,126 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Error processing
datanode Command
> java.io.IOException: Error in deleting blocks.
> 	at org.apache.hadoop.hdfs.server.datanode.FSDataset.invalidate(FSDataset.java:2061)
> 	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:581)
> 	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:545)
> 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:690)
> 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:522)
> 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:662)
> 	at java.lang.Thread.run(Thread.java:619)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message