Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3FB5CF508 for ; Wed, 20 Mar 2013 04:01:07 +0000 (UTC) Received: (qmail 60717 invoked by uid 500); 20 Mar 2013 04:01:02 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 60121 invoked by uid 500); 20 Mar 2013 04:01:01 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 60069 invoked by uid 99); 20 Mar 2013 04:00:59 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Mar 2013 04:00:59 +0000 X-ASF-Spam-Status: No, hits=1.8 required=5.0 tests=HTML_FONT_FACE_BAD,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates 209.85.210.170 as permitted sender) Received: from [209.85.210.170] (HELO mail-ia0-f170.google.com) (209.85.210.170) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Mar 2013 04:00:53 +0000 Received: by mail-ia0-f170.google.com with SMTP id h8so1071514iaa.15 for ; Tue, 19 Mar 2013 21:00:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:content-type:x-gm-message-state; bh=6zqgmeJ4KeV89jA+p6NXqrkMp014+6lLqCaAweWmYD8=; b=pfV6J8zOFpYWNJEDZe65+i8sK7jonkMCHkRMnUNxcJdJbH0sHgRYc3G3uvgYPXoEEJ FOc1Nf9Yx5IooX3f0l4KCtMKLBTwnsx4LC8Z/W3+PH8smt/b/0Im3gjNtTQI/LKukw3x +BXxoPjYfwFGylgwULQbQTF+9oC0wVSxwr4OGbeC1ItHz0a3UXqHcKvlftfKCMbs+c65 Sv/P063xosL9loPEZrFxO/Tkguod4bbK+klC9XHxYdAQ+wZby8SG3ekpqgKkRN55e6SU PH7KUf7sievrclAwU6ScpitrrOVeAfDRKEOx50JrKIe1hjKlar+9DA7zLmcjiywusRPy bO0A== X-Received: by 10.43.103.195 with SMTP id dj3mr12737226icc.3.1363752032577; Tue, 19 Mar 2013 21:00:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.181.198 with HTTP; Tue, 19 Mar 2013 21:00:12 -0700 (PDT) In-Reply-To: References: From: Harsh J Date: Wed, 20 Mar 2013 09:30:12 +0530 Message-ID: Subject: Re: Need your help with Hadoop To: =?UTF-8?B?5aea5ZCJ6b6Z?= , "" Content-Type: multipart/alternative; boundary=bcaec5171a2376cf3604d8534364 X-Gm-Message-State: ALoCoQlnhydmIUtGU2C71XYCaY5XToH5onM0d8w0q9giDd6lbRSc4bFhHKq7qH++QEqslbQYwfa5 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec5171a2376cf3604d8534364 Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: quoted-printable Hi, The settings property is "dfs.data.dir" (or "dfs.datanode.data.dir") and its present in the hdfs-site.xml file at each DataNode, under $HADOOP_HOME/conf/ directories usually. Look for asymmetrical configs among various DNs for that property. On Tue, Mar 19, 2013 at 9:09 PM, =D2=A6=BC=AA=C1=FA = wrote: > Thank for your reply. > I am wondering which parameters defines the capacity of datanode,or the > way to calculate the capacity. I have considered the your answer before, = I > do not know how to modify the settings. > Besides, from my point, the capacity will be related with the disk volum= e > which means that the capacity will be defined by the disk mounted on file > system of Hadoop user's temp directory. While I can't find the detailed > instructions about this. > Why the capacity of others nodes is about 50G? > These bothers me a lot. > > BRs > Geelong > > 2013/3/19 Harsh J > >> You'd probably want to recheck your configuration of dfs.data.dir on the >> node16 (perhaps its overriding the usual default), to see if it is perha= ps >> including more dirs than normal (and they may be all on the same disks a= s >> well, the DN counts space via du/df on each directory so the number can >> grow that way). >> >> Also, please direct usage questions to user@hadoop.apache.org community, >> which I've included in my response :) >> >> >> On Tue, Mar 19, 2013 at 5:40 PM, =D2=A6=BC=AA=C1=FA wrote: >> >>> Hi >>> >>> I am a newer for the Hadoop platform, I really need your help. >>> Now we have 32 datanodes available, while we find that the Configured >>> Capacity is different among these datanodes though the hardware is the = same. >>> I wonder the reson why the node16 is much bigger than the others, >>> besides which is main factor or directory that determine the capacity f= or >>> each datanode. >>> >>> >>> I wiil apprecite your kindly help, this problem has been puzzled me for >>> a long time. >>> >>> BRs >>> Geelong >>> >>> -- >>> From Good To Great >>> >> >> >> >> -- >> Harsh J >> > > > > -- > From Good To Great > --=20 Harsh J --bcaec5171a2376cf3604d8534364 Content-Type: text/html; charset=GB2312 Content-Transfer-Encoding: quoted-printable
Hi,

The settings property is "dfs.= data.dir" (or "dfs.datanode.data.dir") and its present in th= e hdfs-site.xml file at each DataNode, under $HADOOP_HOME/conf/ directories= usually. Look for asymmetrical configs among various DNs for that property= .


On Tue,= Mar 19, 2013 at 9:09 PM, =D2=A6=BC=AA=C1=FA <geelongyao@gmail.com= > wrote:
Thank for your reply.
I am wondering &nb= sp;which parameters defines the capacity of datanode,or the way to cal= culate the capacity. I have considered the your answer before, I do no= t know how to modify the settings.
Besides, from my point, the capacity will be related with the disk volume w= hich means that the capacity will be defined by the disk mounted on fi= le system of Hadoop user's temp directory. While I can't find = the detailed instructions about this.
Why the capacity of others nodes is about 50G?
These bothers= me a lot.

BRs
Geelong

2013/3/19= Harsh J <harsh@cloudera.com>
You'd probably want to = recheck your configuration of dfs.data.dir on the node16 (perhaps its overr= iding the usual default), to see if it is perhaps including more dirs than = normal (and they may be all on the same disks as well, the DN counts space = via du/df on each directory so the number can grow that way).

Also, please direct usage questions to user@hadoop.apache.org community= , which I've included in my response :)


On Tue, Mar 19, 2013 at 5:40 PM, =D2=A6=BC=AA=C1= =FA <geelongyao@gmail.com> wrote:
Hi

I am a newer for the Hadoop platform= , I really need your help.
Now we have 32 datanodes available, wh= ile we find that the Configured Capacity is different among these datanodes= though the hardware is the same.
I wonder the reson why the node16 is much bigger than the others, besi= des which is main factor or directory that determine the capacity for each = datanode.


I wiil apprecite your k= indly help, this problem has been puzzled me for a long time.

BRs
Geelong
=

--
From Good To Great



<= font color=3D"#888888">--
Harsh J



--
From Good To Great



--
= Harsh J
--bcaec5171a2376cf3604d8534364--