hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Bockelman <bbock...@math.unl.edu>
Subject Hit a roadbump in solving truncated block issue
Date Tue, 16 Dec 2008 18:22:31 GMT

I hit a bit of a roadbump in solving the "truncated block issue" at  
our site: namely, some of the blocks appear perfectly valid to the  
datanode.  The block verifies, but it is still the wrong size (it  
appears that the metadata is too small too).

What's the best way to proceed?  It appears that either (a) the block  
scanner needs to report to the datanode the size of the block it just  
verified, which is possibly a scaling issue or (b) the metadata file  
needs to save the correct block size, which is a pretty major  
modification, as it requires a change of the on-disk format.


View raw message