Return-Path: 
 <user-return-4056-apmail-hadoop-hdfs-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id E6804EBAB
	for <apmail-hadoop-hdfs-user-archive@minotaur.apache.org>;
 Wed,  9 Jan 2013 15:26:17 +0000 (UTC)
Received: (qmail 13322 invoked by uid 500); 9 Jan 2013 15:26:13 -0000
Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org
Received: (qmail 13197 invoked by uid 500); 9 Jan 2013 15:26:13 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Delivered-To: moderator for user@hadoop.apache.org
Received: (qmail 61004 invoked by uid 99); 9 Jan 2013 14:40:40 -0000
X-ASF-Spam-Status: No, hits=1.7 required=5.0
	tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of dora0009@gmail.com designates
 209.85.217.170 as permitted sender)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=1HAteNQ/+v4MGdnEE1/fjn4P85OS0dxQQrLPn8GAvxI=;
        b=gc1MSauE9t+QPAffJz/qdI4P9pi4JKnQ0or8V5qUXU/htsv0uCi7VcwUYNn/5IW9ys
         cCIhi7S13Nb6q+NOPrFj5jmSV8s5rJp8c8mY7x9jNpcyPtgvVovTP0TkacvFiNnXcxgD
         kce/JkSaDcY0dbrS8a5r9em3YV8OdaHLDWyTVtTisUxbhJohM75no7kZkN4/ioCh6jXn
         Mn/wGjD9iJ+ZLkNfZLp8l7B5ra3vOkUHdcYN0MTgCOR3nEAJ7qNf7H75g8jR1dXQ7eJc
         2cTwTn750x5RcTkGXuTlsLgAgClacN+kILNVrl2eOV76ozLDk5q/BxGwL7YyDFUPSeqP
         y/3Q==
MIME-Version: 1.0
In-Reply-To: 
 <CACHrRQ+k_86RqVk45qvYc9TJGQDi6QjF4AsAA9ufwSxv525=LA@mail.gmail.com>
References: 
 <CACHrRQ+k_86RqVk45qvYc9TJGQDi6QjF4AsAA9ufwSxv525=LA@mail.gmail.com>
Date: Wed, 9 Jan 2013 22:40:13 +0800
Message-ID: 
 <CACHrRQJqEUNBXmnY-y2uhBeE_0FTZttMi0w-BhzPUfJSJ9tz_w@mail.gmail.com>
Subject: Fwd: How Hadoop decide the capacity of each node
From: Dora <dora0009@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=f46d0401fa8b3f280204d2dc0a41
X-Virus-Checked: Checked by ClamAV on apache.org

--f46d0401fa8b3f280204d2dc0a41
Content-Type: text/plain; charset=ISO-8859-1

Hi Tariq,

Thanks for your answer!
It's very useful for me.
Thanks again :)

Best Regards
Dora
-----------------------------------------------------
Hello Dora,

        As Jean has said, HDFS uses all of your disk by default. You
can though limit the amount of disk you want your DN to use through
the "dfs.datanode.du.reserved" property. The no. of Bytes specified
as the value of this property will be left for non HDFS use.

HTH

Warm Regards,
Tariq
https://mtariq.jux.com/


On Wed, Jan 9, 2013 at 7:15 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:

> Hi Dora,
>
> Hadoop is not deciding. It's "simply" pushing the same amount of data
> on each node. If a node is out of space, it's removed from the "write"
> list and is used only for reads.
>
> Hadoop is only using the space it needs. So if it uses only 50G it's
> because it don't need the extra 50G yet.
>
> JM
>
> 2013/1/9, Dora <dora0009@gmail.com>:
> > Hi all,
> >
> > Could you tell me how Hadoop decide tha capacity of each datanodes?
> > I've installed CDH3 on 2 VM machine, each VM has 100G space,
> > And I found that Hadoop occupied 50G/100G, why?
> > Thanks.
> >
> > Best Regards,
> > Dora
> >
>

--f46d0401fa8b3f280204d2dc0a41
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"moz-text-html" lang=3D"x-western"><div dir=3D"ltr">Hi Tariq,<=
/div><div dir=3D"ltr"><br></div><div dir=3D"ltr">Thanks for your answer!</d=
iv><div dir=3D"ltr">It&#39;s very useful for me.</div><div dir=3D"ltr">Than=
ks again :)</div>
<div dir=3D"ltr"><br></div><div dir=3D"ltr">Best Regards</div><div dir=3D"l=
tr">Dora</div><div dir=3D"ltr">--------------------------------------------=
---------</div><div dir=3D"ltr">Hello Dora,<div><br></div><div>=A0 =A0 =A0 =
=A0 As Jean has said, HDFS uses all of your disk by default.=A0You=A0</div>
<div>can though limit the amount of disk you want your DN to use through</d=
iv><div>

the=A0&quot;dfs.datanode.du.reserved&quot; property. The no. of Bytes speci=
fied</div><div>as the value of this property will be left for non HDFS use.=
</div><div><br></div><div>HTH</div><div class=3D"gmail_extra">

<br clear=3D"all"><div><div dir=3D"ltr">Warm Regards,<div>Tariq</div><div><=
a href=3D"https://mtariq.jux.com/" target=3D"_blank">https://mtariq.jux.com=
/</a><br></div></div></div>
<br><br><div class=3D"gmail_quote">On Wed, Jan 9, 2013 at 7:15 PM, Jean-Mar=
c Spaggiari <span dir=3D"ltr">&lt;<a href=3D"mailto:jean-marc@spaggiari.org=
" target=3D"_blank">jean-marc@spaggiari.org</a>&gt;</span> wrote:<br><block=
quote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc=
 solid;padding-left:1ex">


Hi Dora,<br>
<br>
Hadoop is not deciding. It&#39;s &quot;simply&quot; pushing the same amount=
 of data<br>
on each node. If a node is out of space, it&#39;s removed from the &quot;wr=
ite&quot;<br>
list and is used only for reads.<br>
<br>
Hadoop is only using the space it needs. So if it uses only 50G it&#39;s<br=
>
because it don&#39;t need the extra 50G yet.<br>
<br>
JM<br>
<br>
2013/1/9, Dora &lt;<a href=3D"mailto:dora0009@gmail.com">dora0009@gmail.com=
</a>&gt;:<br>
<div class=3D"HOEnZb"><div class=3D"h5">&gt; Hi all,<br>
&gt;<br>
&gt; Could you tell me how Hadoop decide tha capacity of each datanodes?<br=
>
&gt; I&#39;ve installed CDH3 on 2 VM machine, each VM has 100G space,<br>
&gt; And I found that Hadoop occupied 50G/100G, why?<br>
&gt; Thanks.<br>
&gt;<br>
&gt; Best Regards,<br>
&gt; Dora<br>
&gt;</div></div></blockquote></div></div></div>

</div>

--f46d0401fa8b3f280204d2dc0a41--