hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer <awittena...@linkedin.com>
Subject Re: is there any way we can limit Hadoop Datanode's disk usage?
Date Wed, 31 Mar 2010 18:09:04 GMT
On 3/30/10 8:12 PM, "steven zhuang" <steven.zhuang.1984@gmail.com> wrote:

> hi, guys,
>                we have some machine with 1T disk, some with 100GB disk,
>                I have this question that is there any means we can limit the
> disk usage of datanodes on those machines with smaller disk?
>                thanks!

You can use dfs.datanode.du.reserved, but be aware that are *no* limits on
mapreduce's usage, other than what you can create with file system quotas.

 I've started recommended creating file system partitions in order to work
around Hadoop's crazy space reservation ideas.

View raw message