hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: How Hadoop decide the capacity of each node
Date Wed, 09 Jan 2013 13:57:41 GMT
Hello Dora,

        As Jean has said, HDFS uses all of your disk by default. You
can though limit the amount of disk you want your DN to use through
the "dfs.datanode.du.reserved" property. The no. of Bytes specified
as the value of this property will be left for non HDFS use.

HTH

Warm Regards,
Tariq
https://mtariq.jux.com/


On Wed, Jan 9, 2013 at 7:15 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:

> Hi Dora,
>
> Hadoop is not deciding. It's "simply" pushing the same amount of data
> on each node. If a node is out of space, it's removed from the "write"
> list and is used only for reads.
>
> Hadoop is only using the space it needs. So if it uses only 50G it's
> because it don't need the extra 50G yet.
>
> JM
>
> 2013/1/9, Dora <dora0009@gmail.com>:
> > Hi all,
> >
> > Could you tell me how Hadoop decide tha capacity of each datanodes?
> > I've installed CDH3 on 2 VM machine, each VM has 100G space,
> > And I found that Hadoop occupied 50G/100G, why?
> > Thanks.
> >
> > Best Regards,
> > Dora
> >
>

Mime
View raw message