hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Drake민영근 <drake....@nexr.com>
Subject HDFS Datanode Blockreport includes invalid block.
Date Thu, 01 Oct 2015 04:56:46 GMT
Greetings, All

Recently, I met the worst case of HDFS, the missing block. The
timeline(log) of namenode is below:

initial state: Datanode A(or B or C. not confirm yet) and
contains block_1.

(Datanode A, B, C crash w/ hardware fault)
07:33:03: ask replication block_1 from to
07:34:14: updatedBlockmap addStoredBlock with block_1
(Datanode A, B, C recover)
18:11:36: block_1 of is invalidate
18:11:45: block_1 of is deleted
19:43:03: updatedBlockmap addStoredBlock with block_1
19:43:03: block_1 of is invalidate
19:43:04: block_1 of is deleted
(Datanode A, B, C crash w/ hardware fault AGAIN)
00:27:21: ask replication block_1 from to
07:34:14: Error cause block_1 of is invalid

At 19:43:03, Datanode send the block report to namenode. I
guess's block report contains worng, in this case invalid,

Anyone seen this problem ?

Sorry for log format. I cannot get the full logs.


Drake 민영근 Ph.D
kt NexR

View raw message