hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yu Li <car...@gmail.com>
Subject Re: Question about disk space allocation in hadoop
Date Wed, 30 Jun 2010 03:10:48 GMT
Hi all,

Anybody has experience on this? Any Comments/Suggestions would be
highly appreciated, Thanks.

Best Regards,

2010/6/29 Yu Li <carp84@gmail.com>:
> Hi all,
> As we all know, machines in hadoop cluster may be both datanode and
> tasktracker, so one machine may store both MR job intermediate data
> and HDFS data. My question is: if we have more than one disk per node,
> say 4 disks, and would like both job intermediate data and HDFS data
> store into all disks to reduce IO times of each single disk, can we
> draw a line between space of local FS and HDFS? For example, restrict
> the intermediate temp data occupy no more than 25% space on each disk?
> Thanks in advance.
> Best Regards,
> Carp

View raw message