Return-Path: Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: (qmail 41200 invoked from network); 16 Mar 2011 06:05:13 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 16 Mar 2011 06:05:13 -0000 Received: (qmail 62321 invoked by uid 500); 16 Mar 2011 06:05:10 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 62230 invoked by uid 500); 16 Mar 2011 06:05:09 -0000 Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-user@hadoop.apache.org Delivered-To: mailing list common-user@hadoop.apache.org Received: (qmail 62222 invoked by uid 99); 16 Mar 2011 06:05:09 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Mar 2011 06:05:09 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RFC_ABUSE_POST,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [98.139.91.198] (HELO nm10-vm0.bullet.mail.sp2.yahoo.com) (98.139.91.198) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 16 Mar 2011 06:04:58 +0000 Received: from [98.139.91.63] by nm10.bullet.mail.sp2.yahoo.com with NNFMP; 16 Mar 2011 06:04:37 -0000 Received: from [98.139.91.54] by tm3.bullet.mail.sp2.yahoo.com with NNFMP; 16 Mar 2011 06:04:37 -0000 Received: from [127.0.0.1] by omp1054.mail.sp2.yahoo.com with NNFMP; 16 Mar 2011 06:04:37 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 190830.49534.bm@omp1054.mail.sp2.yahoo.com Received: (qmail 78897 invoked by uid 60001); 16 Mar 2011 06:04:36 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1300255476; bh=ikLH9aWE4AJkUhJbRHNBBcmssyznCBSEp75XuSbS/OQ=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=L2lHxP7GVsusW2tumAKm8TRUEAZSLG/GXhZvQNvT16JnDBWaR0eCe31hYMft0drOfJt1zFZx6Eg89fp58mso6Ba9ui6C0ctuJKtWgHEdkQfZ5r0zTLLEpyBuzevIlEETk5MS7BSd5ZCGsFkQe6L3+sQ1Rjcq3t93wsqx3wzqNTI= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=UcfQWP8WSpuGJGWIpmc8YIeSRF8Hd5Bxe5+ELjAKv/JPhZ3Zhcep1g6xCXwChhXVIY2cx3iZ0FlQ3VuPixkrtHePUXvVXtUWJUuaE+/Z+qq+L+ZOhGxY3G4f+hgckMb7ZsXu/uI8TM3BabodQUXdQr4VWnaMBt7/tPhka1UQ5HQ=; Message-ID: <880550.24416.qm@web110710.mail.gq1.yahoo.com> X-YMail-OSG: 2aO_PVYVM1ln7QBj12i74ZCEE0.338rY.wc9uy9CdwUdP93 UN6lyZsJ.VlG4nApRqZb8hikZ956pST8hpYcqBa0dXX8JOdsLNyrcHOHP9wo 0X9mTfBxqh95X7teAQgO4oGhdaZFO1KwgEVaGXor159CLgeKn7tnUY2Ojgmp qtra_0.WGRIYIMtKMlnceyzWxMMNUFyvilEsUZbBJPbw1aHfSbobhr2N.4F6 6n9jUgKA_8lYd4R9DgvtokSTg4LrFZwYnE8.fiZ5N3IPGBEN_TTK0dlU0YN_ 4I7jVjBCSVRVgsB5qpZaKvX3QNQStzMcAw6XyJYltOSkwH5fzICKn7f53SjB UgeJuIdBszO0uAI7izeJqT9Msd9k9WALYv0VYTqCHQT_0_lZvzQvqC1eiMXE wfpeOBbUQ3wRdxEQ- Received: from [209.131.62.115] by web110710.mail.gq1.yahoo.com via HTTP; Tue, 15 Mar 2011 23:04:36 PDT X-Mailer: YahooMailWebService/0.8.109.295617 Date: Tue, 15 Mar 2011 23:04:36 -0700 (PDT) From: Bharath Mundlapudi Reply-To: Bharath Mundlapudi Subject: Re: Load testing in hadoop To: "common-user@hadoop.apache.org" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-331689981-1300255476=:24416" X-Virus-Checked: Checked by ClamAV on apache.org --0-331689981-1300255476=:24416 Content-Type: text/plain; charset=us-ascii Gridmix v3 and TeraSort are good benchmarks to test overall hadoop cluster. But synthetic load generator is specifically for Namenode. You can refer the documentation below for how to run this tool. http://hadoop.apache.org/common/docs/r0.20.2/SLG_user_guide.html Also, there are various other benchmarks (TestDFSIO, NNBench etc) depending on which area you want to benchmark. -Bharath ________________________________ From: Steve Loughran To: common-user@hadoop.apache.org Sent: Tuesday, March 15, 2011 3:35 AM Subject: Re: Load testing in hadoop On 15/03/11 04:59, Kannu wrote: > > Please tell me how to use synthetic load generator in hadoop or suggest me > any other way of load testing in hadoop cluster. > > thanks, > kannu terasort is the one most people use, as it generates its own datasets. Otherwise you need a few TB of data and some custom code to run on it. Gridmix/gridmix2 can also simulate load. All this stuff is in the hadoop source tree, with documentation --0-331689981-1300255476=:24416--