hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@yahoo-inc.com>
Subject Re: [jira] Updated: (HADOOP-1180) NNbench test should be able to test the checksumfilesystem as well as the raw filesystem
Date Fri, 30 Mar 2007 21:50:01 GMT
Doug Cutting wrote:
> hairong Kuang wrote:
>> 1. NNBench sets the block size to be 1. Althouth it generates a file with
>> only 1 byte,  but the file's checksum file has 16 bytes (12 bytes header
>> plus 4 bytes checksums). Without the checksum file, only 1 block needs 
>> to be
>> generated. With the checksum file, 17 blocks need to be generated. So the
>> overhead of generating a checksum file is huge in this special case.
> 
> So to make this benchmark more representative of real performance with 
> lots of small files, we should change the block size to 16 or greater. 
> While small files may be typical, a blocksize of 1 is not.  Since the 
> benchmark only writes one byte per file, a tiny block size doesn't 
> really need to be set at all for this benchmark.

+1. I was thinking of the same. We could even leave the block size to 
default and create 1 byte files as we do now.

Raghu.

> Doug


Mime
View raw message