Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8D62510BBD for ; Wed, 5 Mar 2014 07:48:19 +0000 (UTC) Received: (qmail 76797 invoked by uid 500); 5 Mar 2014 07:48:09 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 75069 invoked by uid 500); 5 Mar 2014 07:48:06 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 75057 invoked by uid 99); 5 Mar 2014 07:48:04 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Mar 2014 07:48:04 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of divs.sheth@gmail.com designates 209.85.214.169 as permitted sender) Received: from [209.85.214.169] (HELO mail-ob0-f169.google.com) (209.85.214.169) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Mar 2014 07:47:59 +0000 Received: by mail-ob0-f169.google.com with SMTP id va2so658272obc.28 for ; Tue, 04 Mar 2014 23:47:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=xwQc9WFuHYLP7UQLitGwJNeCQJADGEQi3fB3/OC2kP4=; b=XtZhqK+c+oXjknMsvshQBcNkA/pRBfE1T+OeWbNZRIl9/5NAqpME0w1vNj4vN7fLfj eTiz+Tih/HikghKv5l2258iFLn1pP9Obk2/olOLJdK1b6BBioBFLI0JKE6R65nEBpBus bIq8jYZnVJSOgniYnfFe+b84b+zYMo544qylPPT0idJEfsSkuc3uMLPxKPi7X1ryE7/z lDrAKdE3P9XyvjXn4miI+Z7M40Npx6yLYNh///Qe963WHz5b8gqtCT7s59xXR0LesEGU MP/ieWpCI38vRSrL33awLta5aumNyllj5Yyc2oDB+ZKgDE66vxv/zcfeLkSfJQvfsWoB Lbpg== MIME-Version: 1.0 X-Received: by 10.182.34.196 with SMTP id b4mr3451795obj.13.1394005658224; Tue, 04 Mar 2014 23:47:38 -0800 (PST) Received: by 10.76.23.4 with HTTP; Tue, 4 Mar 2014 23:47:38 -0800 (PST) In-Reply-To: References: Date: Wed, 5 Mar 2014 13:17:38 +0530 Message-ID: Subject: Re: Question on DFS Balancing From: divye sheth To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11c2cba412cd2304f3d73c2a X-Virus-Checked: Checked by ClamAV on apache.org --001a11c2cba412cd2304f3d73c2a Content-Type: text/plain; charset=ISO-8859-1 Thanks Harsh. The jira is fixed in version 2.1.0 whereas I am using Hadoop 0.20.2 (we are in a process of upgrading) is there a workaround for the short term to balance the disk utilization? The patch in the Jira, if applied to the version that I am using, will it break anything? Thanks Divye Sheth On Wed, Mar 5, 2014 at 11:28 AM, Harsh J wrote: > You're probably looking for > https://issues.apache.org/jira/browse/HDFS-1804 > > On Tue, Mar 4, 2014 at 5:54 AM, divye sheth wrote: > > Hi, > > > > I am new to the mailing list. > > > > I am using Hadoop 0.20.2 with an append r1056497 version. The question I > > have is related to balancing. I have a 5 datanode cluster and each node > has > > 2 disks attached to it. The second disk was added when the first disk was > > reaching its capacity. > > > > Now the scenario that I am facing is, when the new disk was added hadoop > > automatically moved over some data to the new disk. But over the time I > > notice that data is no longer being written to the second disk. I have > also > > faced an issue on the datanode where the first disk had 100% utilization. > > > > How can I overcome such scenario, is it not hadoop's job to balance the > disk > > utilization between multiple disks on single datanode? > > > > Thanks > > Divye Sheth > > > > -- > Harsh J > --001a11c2cba412cd2304f3d73c2a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thanks Harsh. The jira is fixed in version 2.1.0 whereas I= am using Hadoop 0.20.2 (we are in a process of upgrading) is there a worka= round for the short term to balance the disk utilization? The patch in the = Jira, if applied to the version that I am using, will it break anything?
Thanks
Divye Sheth


On Wed, Mar 5, 2014 at 11:28 AM, = Harsh J <harsh@cloudera.com> wrote:
You're probably looking for https://i= ssues.apache.org/jira/browse/HDFS-1804

On Tue, Mar 4, 2014 at 5:54 AM, divye sheth <divs.sheth@gmail.com> wrote:
> Hi,
>
> I am new to the mailing list.
>
> I am using Hadoop 0.20.2 with an append r1056497 version. The question= I
> have is related to balancing. I have a 5 datanode cluster and each nod= e has
> 2 disks attached to it. The second disk was added when the first disk = was
> reaching its capacity.
>
> Now the scenario that I am facing is, when the new disk was added hado= op
> automatically moved over some data to the new disk. But over the time = I
> notice that data is no longer being written to the second disk. I have= also
> faced an issue on the datanode where the first disk had 100% utilizati= on.
>
> How can I overcome such scenario, is it not hadoop's job to balanc= e the disk
> utilization between multiple disks on single datanode?
>
> Thanks
> Divye Sheth



--
Harsh J

--001a11c2cba412cd2304f3d73c2a--