hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dennis Kubes <nutch-...@dragonflymc.com>
Subject Re: Benchmarking question - how do you test a new cluster?
Date Mon, 23 Apr 2007 18:00:05 GMT
You will also want to run hardware tests on the machines in your cluster 
to make sure memory, disk, and network is working properly.  We use 
tools such as memtest86+ and Doug Ledfors Memory Test Script to do burn 
ins.  Here is a link to different test programs for linux.


Dennis Kubes

Owen O'Malley wrote:
> On Apr 23, 2007, at 7:39 AM, Steve Schlosser wrote:
>> I've got a small hadoop cluster running (5 nodes today, going to 15+
>> soon), and I'd like to do some benchmarking.  My question to the group
>> is - what is the first benchmark you run on a new cluster?
> I usually use random-writer to generate some random data (it defaults to 
> 10g/node) and then use sort to sort it. Sort provides a pretty decent 
> simple testcase for moving a lot of data through map/reduce.
> -- Owen

View raw message