hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharath Mundlapudi <bharathw...@yahoo.com>
Subject Re: Load testing in hadoop
Date Wed, 16 Mar 2011 06:04:36 GMT
Gridmix v3 and TeraSort are good benchmarks to test overall hadoop cluster. But synthetic load
generator is specifically for Namenode. You can refer the documentation below for how to run
this tool.


Also, there are various other benchmarks (TestDFSIO, NNBench etc) depending on which area
you want to benchmark.


From: Steve Loughran <stevel@apache.org>
To: common-user@hadoop.apache.org
Sent: Tuesday, March 15, 2011 3:35 AM
Subject: Re: Load testing in hadoop

On 15/03/11 04:59, Kannu wrote:
> Please tell me how to use synthetic load generator in hadoop or suggest me
> any other way of load testing in hadoop cluster.
> thanks,
> kannu

terasort is the one most people use, as it generates its own datasets. 
Otherwise you need a few TB of data and some custom code to run on it.

Gridmix/gridmix2 can also simulate load.

All this stuff is in the hadoop source tree, with documentation

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message