hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yu Li <car...@gmail.com>
Subject Question about disk space allocation in hadoop
Date Tue, 29 Jun 2010 04:32:25 GMT
Hi all,

As we all know, machines in hadoop cluster may be both datanode and
tasktracker, so one machine may store both MR job intermediate data
and HDFS data. My question is: if we have more than one disk per node,
say 4 disks, and would like both job intermediate data and HDFS data
store into all disks to reduce IO times of each single disk, can we
draw a line between space of local FS and HDFS? For example, restrict
the intermediate temp data occupy no more than 25% space on each disk?
Thanks in advance.

Best Regards,
Carp

Mime
View raw message