hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vitaliy Semochkin <vitaliy...@gmail.com>
Subject Re: Question about disk space allocation in hadoop
Date Wed, 30 Jun 2010 10:18:53 GMT
set dfs.datanode.du.reserved to amount of bytes you want to reserver for not
HDFS usage.

for search convenience IMHO better post such questions to
hdfs-user@hadoop.apache.org ;-)

Vitaliy S

On Tue, Jun 29, 2010 at 8:32 AM, Yu Li <carp84@gmail.com> wrote:

> Hi all,
> As we all know, machines in hadoop cluster may be both datanode and
> tasktracker, so one machine may store both MR job intermediate data
> and HDFS data. My question is: if we have more than one disk per node,
> say 4 disks, and would like both job intermediate data and HDFS data
> store into all disks to reduce IO times of each single disk, can we
> draw a line between space of local FS and HDFS? For example, restrict
> the intermediate temp data occupy no more than 25% space on each disk?
> Thanks in advance.
> Best Regards,
> Carp

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message