hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharath Mundlapudi <bharathw...@yahoo.com>
Subject Re: Load testing in hadoop
Date Wed, 16 Mar 2011 06:04:36 GMT
Gridmix v3 and TeraSort are good benchmarks to test overall hadoop cluster. But synthetic load
generator is specifically for Namenode. You can refer the documentation below for how to run
this tool.

http://hadoop.apache.org/common/docs/r0.20.2/SLG_user_guide.html 


Also, there are various other benchmarks (TestDFSIO, NNBench etc) depending on which area
you want to benchmark.

-Bharath





________________________________
From: Steve Loughran <stevel@apache.org>
To: common-user@hadoop.apache.org
Sent: Tuesday, March 15, 2011 3:35 AM
Subject: Re: Load testing in hadoop

On 15/03/11 04:59, Kannu wrote:
>
> Please tell me how to use synthetic load generator in hadoop or suggest me
> any other way of load testing in hadoop cluster.
>
> thanks,
> kannu

terasort is the one most people use, as it generates its own datasets. 
Otherwise you need a few TB of data and some custom code to run on it.

Gridmix/gridmix2 can also simulate load.

All this stuff is in the hadoop source tree, with documentation


      
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message