hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-39) Namenode blockMap not updated when datanode invalidates a block on heart beat
Date Wed, 19 Mar 2014 22:49:48 GMT

     [ https://issues.apache.org/jira/browse/HDFS-39?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tsz Wo Nicholas Sze resolved HDFS-39.
-------------------------------------

    Resolution: Not A Problem

Anyway, this issue is "Not A Problem" anymore.  Resolving ...

> Namenode blockMap not updated when datanode invalidates a block on heart beat
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-39
>                 URL: https://issues.apache.org/jira/browse/HDFS-39
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Wei Xu
>
> Here are related logs.  dn_10_251_107_213 deleted the block immediately after it received
it (because of over-replication).  However, the information is not reported to namenode and
name node ask this DN to invalidate the block again, causing the error "Unexpected error trying
to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.". 
> dn_10_251_107_213 | 080722 145554 2497 INFO dfs.DataNode$DataXceiver: Receiving block
blk_-1201258788540100829 src: /10.251.67.214:54770 dest: /10.251.67.214:50010
> dn_10_251_193_205 | 080722 145554 2468 INFO dfs.DataNode$DataXceiver: Receiving block
blk_-1201258788540100829 src: /10.251.107.213:36718 dest: /10.251.107.213:50010
> dn_10_251_67_214 | 080722 145554 2306 INFO dfs.DataNode$DataXceiver: Receiving block
blk_-1201258788540100829 src: /10.251.67.214:60599 dest: /10.251.67.214:50010
> nn_10_251_210_161 | 080722 145554 30 INFO dfs.FSNamesystem: BLOCK* NameSystem.allocateBlock:
/user/root/rand/_temporary/_task_200807221347_0001_m_000092_1/part-00092. blk_-1201258788540100829
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: Received block
blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: Received block
blk_-1201258788540100829 of size 29767698 from /10.251.107.213
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: Received block
blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: PacketResponder
1 for block blk_-1201258788540100829 terminating
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: PacketResponder
0 for block blk_-1201258788540100829 terminating
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: PacketResponder
2 for block blk_-1201258788540100829 terminating
> nn_10_251_210_161 | 080722 145656 19 INFO dfs.FSNamesystem: BLOCK* ask 10.251.67.214:50010
to replicate blk_-1201258788540100829 to datanode(s) 10.251.107.49:50010
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock:
blockMap updated: 10.251.67.214:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock:
blockMap updated: 10.251.107.213:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 26 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock:
blockMap updated: 10.251.193.205:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_49 | 080722 145658 2484 INFO dfs.DataNode$DataXceiver: Receiving block
blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010
> dn_10_251_67_214 | 080722 145658 18 INFO dfs.DataNode: 10.251.67.214:50010 Starting thread
to transfer block blk_-1201258788540100829 to 10.251.107.49:50010
> dn_10_251_107_49 | 080722 145705 2484 INFO dfs.DataNode$DataXceiver: Received block blk_-1201258788540100829
src: /10.251.67.214:42237 dest: /10.251.67.214:50010 of size 29767698
> dn_10_251_67_214 | 080722 145705 2342 INFO dfs.DataNode$DataTransfer: 10.251.67.214:50010:Transmitted
block blk_-1201258788540100829 to /10.251.107.49:50010
> nn_10_251_210_161 | 080722 145705 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock:
blockMap updated: 10.251.107.49:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_213 | 080722 145710 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829
file /mnt/hadoop/dfs/data/current/subdir11/blk_-1201258788540100829
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete:
blk_-1201258788540100829 is added to invalidSet of 10.251.67.214:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete:
blk_-1201258788540100829 is added to invalidSet of 10.251.107.213:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete:
blk_-1201258788540100829 is added to invalidSet of 10.251.193.205:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete:
blk_-1201258788540100829 is added to invalidSet of 10.251.107.49:50010
> dn_10_251_193_205 | 080722 150224 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829
file /mnt/hadoop/dfs/data/current/subdir51/blk_-1201258788540100829
> dn_10_251_107_49 | 080722 150226 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829
file /mnt/hadoop/dfs/data/current/subdir24/blk_-1201258788540100829
> dn_10_251_107_213 | 080722 150228 19 WARN dfs.FSDataset: Unexpected error trying to delete
block blk_-1201258788540100829. BlockInfo not found in volumeMap.
> dn_10_251_67_214 | 080722 150253 18 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829
file /mnt/hadoop/dfs/data/current/subdir43/blk_-1201258788540100829



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message