Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 28E4E17F1B for ; Wed, 12 Nov 2014 00:13:15 +0000 (UTC) Received: (qmail 96234 invoked by uid 500); 12 Nov 2014 00:13:10 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 96105 invoked by uid 500); 12 Nov 2014 00:13:10 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 96095 invoked by uid 99); 12 Nov 2014 00:13:10 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Nov 2014 00:13:10 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of premal.j.shah@gmail.com designates 209.85.160.179 as permitted sender) Received: from [209.85.160.179] (HELO mail-yk0-f179.google.com) (209.85.160.179) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Nov 2014 00:12:43 +0000 Received: by mail-yk0-f179.google.com with SMTP id 131so545822ykp.10 for ; Tue, 11 Nov 2014 16:12:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=i2Voy3wMLm3/T8N9tSC3QL4xQPSq1NoKJQeR8ZAi+II=; b=e5nQHqGJrGRNnfqnC9C3eg+Up6KicoHeA5NVrDib3Pdu4eXWaRHx6PM2p8eR3g/PCb s4tMIhBdJabjQa966QUDYL+wALtS1IREBXT+a+kmz59QIU6HilON07kNAf9HpTQcWsAh DZ/+07pUNh4ji0UpWNeuCz6X7xJ1O5eMFsPP/ttMkq8+1ybtetgEbEsMVW/1Emv7Vnmf 6GGFHqrCjsK440rk8e/uz7qilPmLxD3ABmxCKJOW2jMmmG5J3+y41PHpWQPvqrlL5+uL AjSXDVfEDemQvhwg9MuR88wKt4Sikkv8DhC881qy2NP1zle2wHAtv22T97JN5ALrDo6K AV+w== MIME-Version: 1.0 X-Received: by 10.236.19.69 with SMTP id m45mr40152294yhm.111.1415751162234; Tue, 11 Nov 2014 16:12:42 -0800 (PST) Received: by 10.170.229.3 with HTTP; Tue, 11 Nov 2014 16:12:42 -0800 (PST) In-Reply-To: References: Date: Tue, 11 Nov 2014 16:12:42 -0800 Message-ID: Subject: Re: From: Premal Shah To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=089e0160d2cc1d8d3c05079e4116 X-Virus-Checked: Checked by ClamAV on apache.org --089e0160d2cc1d8d3c05079e4116 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I'm sorry, I send this email without a subject. This is from iostat avg-cpu: %user %nice %system %iowait %steal %idle 1.98 0.00 0.34 0.71 0.02 96.94 When we run MR jobs, I see bandwidth usage spike to 1Gbps easily. But when there are no jobs running, balancer is not using all the network resources. On Tue, Nov 11, 2014 at 3:34 PM, daemeon reiydelle wrote: > I observed what I thought was a similar problem but found that I was > actually hitting the physical disks much harder/more efficiently ... I > found that I had some spindles with pretty deep queue depths. When I sat > back and actually measured, I found that my throughput had substantially > increased, moving the bottleneck to the disks. What does iostats tell you= ? > > > > > > > > > > *.......=E2=80=9CThe race is not to the swift,nor the battle to the stron= g,but to > those who can see it coming and jump aside.=E2=80=9D - Hunter ThompsonDae= meon C.M. > ReiydelleUSA (+1) 415.501.0198 <%28%2B1%29%20415.501.0198>London (+44) (0= ) > 20 8144 9872 <%28%2B44%29%20%280%29%2020%208144%209872>* > > On Tue, Nov 11, 2014 at 2:43 PM, Premal Shah > wrote: > >> Hi, >> We recently upgraded from 1.0.4 to 1.2.1 and now have a ton of DFS >> balancing to do. >> On 1.0.4, when I set the bandwidth to 1 or 10 Gbps, the datanodes got >> working and were using a ton of network to get the nodes balanced. Howev= er, >> looks like 1.2.1 is not honoring the setBandWidth setting, no matter how >> high I set it to be. The datanodes are using only around 30Mbps. Is ther= e a >> way to speed up the balancing? >> >> -- >> Regards, >> Premal Shah. >> > > --=20 Regards, Premal Shah. --089e0160d2cc1d8d3c05079e4116 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I'm sorry, I send this email without a subject.
This is from iostat

avg-cpu: =C2= =A0%user =C2=A0 %nice %system %iowait =C2=A0%steal =C2=A0 %idle
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A01.98 =C2=A0 =C2=A00.00 =C2=A0 =C2= =A00.34 =C2=A0 =C2=A00.71 =C2=A0 =C2=A00.02 =C2=A0 96.94
When we run MR jobs, I see bandwidth usage spike to 1Gbps easi= ly. But when there are no jobs running, balancer is not using all the netwo= rk resources.

On Tue, Nov 11, 2014 at 3:34 PM, daemeon reiydelle <daemeonr@gmai= l.com> wrote:
I observed what I thought was a similar problem but= found that I was actually hitting the physical disks much harder/more effi= ciently ... I found that I had some spindles with pretty deep queue depths.= When I sat back and actually measured, I found that my throughput had subs= tantially increased, moving the bottleneck to the disks. What does iostats = tell you?

=

.......
=E2=80=9CThe race is not to= the swift,
nor the battle to the strong,
but to those who can see it= coming and jump aside.=E2=80=9D - Hunter Thompson

Daemeon C.M. Reiydelle
USA (+1) 415.5= 01.0198
London (+44) (0) 20 8144 9872
=

On Tue, Nov 11, 2014 at 2:43 PM, Premal Shah= <premal.j.shah@gmail.com> wrote:
Hi,
We recently upgraded from 1.0.4 to 1= .2.1 and now have a ton of DFS balancing to do.
On 1.0.4, when I set the bandwid= th to 1 or 10 Gbps, the datanodes got working and were using a ton of netwo= rk to get the nodes balanced. However, looks like 1.2.1 is not honoring the= setBandWidth setting, no matter how high I set it to be. The datanodes are= using only around 30Mbps. Is there a way to speed up the balancing? =C2=A0=

--
Regards,
Premal Shah.




--
Regards,
Premal Shah.
--089e0160d2cc1d8d3c05079e4116--