hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew E. Kennedy" <matt.kenn...@spadac.com>
Subject DataBlockScanner writing hundreds of msgs/sec
Date Thu, 06 Jan 2011 17:58:28 GMT

I have just come across a problem that I can't seem to find much info on in The Google.

I have one slave node (v 0.20.1) that is repeatedly filling up one of its disks with the same
log message from the DataBlockScanner that reads:

2011-01-06 04:03:26,743 INFO org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
failed for blk_-*_*. Its ok since it not in datanode dataset anymore.

The same error gets written to the log file hundreds of times per second, filling up the disk
pretty quickly.  

Is there a polite way to expunge a block from the system other than just deleting it on the
filesystem?  I checked the code, and it looks like it attempts to delete a block like this
under normal circumstances, but for whatever reason, this block isn't going away.

View raw message