hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4556) Block went missing
Date Fri, 31 Oct 2008 21:59:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644439#action_12644439
] 

dhruba borthakur commented on HADOOP-4556:
------------------------------------------

Ok, got it. This is a race between the block report in Step 3 ("Now P is marked dead and P
comes back up again") and the block-delete-call that the namenode issued in Step 1. The block
report inserted a block in the blocksmap that is going to be deleted from the datanode.

Two options come to mind:

1. On every block report processing, the NN clears out blocks from invalidateSets that belong
to that datanode. This might  consume non-trivial amount of CPU on the namenode.

2. The datanode sends its current time on every block report, let call this as the reportStamp.
The NN stores the reportStamp in the DatanodeDescriptor and sends it back to the DN on every
block deletion message. The Datanode process a delete-block request only if the reportStamp
in the delete-block-message is greater than the reportStamp it used in the last block report
that it send out.




> Block went missing
> ------------------
>
>                 Key: HADOOP-4556
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4556
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.17.2
>            Reporter: Robert Chansler
>            Assignee: Hairong Kuang
>
> Suspicion that all replicas of a block were marked for deletion. (Don't panic, investigation
underway.)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message