hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: [jira] Updated: (HADOOP-1180) NNbench test should be able to test the checksumfilesystem as well as the raw filesystem
Date Fri, 30 Mar 2007 16:09:39 GMT
hairong Kuang wrote:
> 1. NNBench sets the block size to be 1. Althouth it generates a file with
> only 1 byte,  but the file's checksum file has 16 bytes (12 bytes header
> plus 4 bytes checksums). Without the checksum file, only 1 block needs to be
> generated. With the checksum file, 17 blocks need to be generated. So the
> overhead of generating a checksum file is huge in this special case.

So to make this benchmark more representative of real performance with 
lots of small files, we should change the block size to 16 or greater. 
While small files may be typical, a blocksize of 1 is not.  Since the 
benchmark only writes one byte per file, a tiny block size doesn't 
really need to be set at all for this benchmark.


View raw message