hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3885) Namenode blockMap not updated when datanode invalidates a block on heart beat
Date Wed, 17 Dec 2008 23:54:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657622#action_12657622
] 

Hairong Kuang commented on HADOOP-3885:
---------------------------------------

All I meant to say is that what's described in this jira is the expected behavior of dfs.
It does not cause any data lose except for some harmless deletion warnings at datanodes. One
improvement we could do is to make the deletion confirmation back to NN sooner. Currently
it waits for the next block report to send the confirmation back. But in between a block deletion
is scheduled to remove and the deletion confirmation comes back, the reported issue is likely
to happen.

> Namenode blockMap not updated when datanode invalidates a block on heart beat
> -----------------------------------------------------------------------------
>
>                 Key: HADOOP-3885
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3885
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.18.0
>            Reporter: Wei Xu
>
> Here are related logs.  dn_10_251_107_213 deleted the block immediately after it received
it (because of over-replication).  However, the information is not reported to namenode and
name node ask this DN to invalidate the block again, causing the error "Unexpected error trying
to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.". 
> dn_10_251_107_213 | 080722 145554 2497 INFO dfs.DataNode$DataXceiver: Receiving block
blk_-1201258788540100829 src: /10.251.67.214:54770 dest: /10.251.67.214:50010
> dn_10_251_193_205 | 080722 145554 2468 INFO dfs.DataNode$DataXceiver: Receiving block
blk_-1201258788540100829 src: /10.251.107.213:36718 dest: /10.251.107.213:50010
> dn_10_251_67_214 | 080722 145554 2306 INFO dfs.DataNode$DataXceiver: Receiving block
blk_-1201258788540100829 src: /10.251.67.214:60599 dest: /10.251.67.214:50010
> nn_10_251_210_161 | 080722 145554 30 INFO dfs.FSNamesystem: BLOCK* NameSystem.allocateBlock:
/user/root/rand/_temporary/_task_200807221347_0001_m_000092_1/part-00092. blk_-1201258788540100829
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: Received block
blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: Received block
blk_-1201258788540100829 of size 29767698 from /10.251.107.213
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: Received block
blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: PacketResponder
1 for block blk_-1201258788540100829 terminating
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: PacketResponder
0 for block blk_-1201258788540100829 terminating
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: PacketResponder
2 for block blk_-1201258788540100829 terminating
> nn_10_251_210_161 | 080722 145656 19 INFO dfs.FSNamesystem: BLOCK* ask 10.251.67.214:50010
to replicate blk_-1201258788540100829 to datanode(s) 10.251.107.49:50010
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock:
blockMap updated: 10.251.67.214:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock:
blockMap updated: 10.251.107.213:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 26 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock:
blockMap updated: 10.251.193.205:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_49 | 080722 145658 2484 INFO dfs.DataNode$DataXceiver: Receiving block
blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010
> dn_10_251_67_214 | 080722 145658 18 INFO dfs.DataNode: 10.251.67.214:50010 Starting thread
to transfer block blk_-1201258788540100829 to 10.251.107.49:50010
> dn_10_251_107_49 | 080722 145705 2484 INFO dfs.DataNode$DataXceiver: Received block blk_-1201258788540100829
src: /10.251.67.214:42237 dest: /10.251.67.214:50010 of size 29767698
> dn_10_251_67_214 | 080722 145705 2342 INFO dfs.DataNode$DataTransfer: 10.251.67.214:50010:Transmitted
block blk_-1201258788540100829 to /10.251.107.49:50010
> nn_10_251_210_161 | 080722 145705 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock:
blockMap updated: 10.251.107.49:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_213 | 080722 145710 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829
file /mnt/hadoop/dfs/data/current/subdir11/blk_-1201258788540100829
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete:
blk_-1201258788540100829 is added to invalidSet of 10.251.67.214:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete:
blk_-1201258788540100829 is added to invalidSet of 10.251.107.213:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete:
blk_-1201258788540100829 is added to invalidSet of 10.251.193.205:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete:
blk_-1201258788540100829 is added to invalidSet of 10.251.107.49:50010
> dn_10_251_193_205 | 080722 150224 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829
file /mnt/hadoop/dfs/data/current/subdir51/blk_-1201258788540100829
> dn_10_251_107_49 | 080722 150226 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829
file /mnt/hadoop/dfs/data/current/subdir24/blk_-1201258788540100829
> dn_10_251_107_213 | 080722 150228 19 WARN dfs.FSDataset: Unexpected error trying to delete
block blk_-1201258788540100829. BlockInfo not found in volumeMap.
> dn_10_251_67_214 | 080722 150253 18 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829
file /mnt/hadoop/dfs/data/current/subdir43/blk_-1201258788540100829

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message