hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@yahoo-inc.com>
Subject Re: Hit a roadbump in solving truncated block issue
Date Tue, 16 Dec 2008 22:10:20 GMT
Brian Bockelman wrote:
> Hey,
> 
> I hit a bit of a roadbump in solving the "truncated block issue" at our 
> site: namely, some of the blocks appear perfectly valid to the 
> datanode.  The block verifies, but it is still the wrong size (it 
> appears that the metadata is too small too).
> 
> What's the best way to proceed?  It appears that either (a) the block 
> scanner needs to report to the datanode the size of the block it just 
> verified, which is possibly a scaling issue or (b) the metadata file 
> needs to save the correct block size, which is a pretty major 
> modification, as it requires a change of the on-disk format.

This should be detected by the NameNode. i.e. it should detect this 
replica is shorter (either compared to other replicas or the expected 
size). There are various fixes (recent or being worked on) to this area 
of NameNode and it is mostly covered by of those or should be soon.

Raghu.

> Ideas?
> 
> Brian


Mime
View raw message