hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anil gupta <anilgupt...@gmail.com>
Subject Can hadoop.tmp.dir be multivalued?
Date Tue, 18 Dec 2012 18:45:12 GMT
Hi All,

On my worker nodes i have 10 drives. So, in order to balance disk i/o i
wanted to evenly distribute the disk read/write load. "hadoop.tmp.dir" is
used for a lot of things in MR.

mapreduce.cluster.local.dir${hadoop.tmp.dir}/mapred/localThe local
directory where MapReduce stores intermediate data files. May be a
comma-separated list of directories on different devices in order to spread
disk i/o. Directories that do not exist are ignored.
mapreduce.jobtracker.system.dir${hadoop.tmp.dir}/mapred/systemThe directory
where MapReduce stores control files.  mapreduce.jobtracker.staging.root.dir
${hadoop.tmp.dir}/mapred/stagingThe root of the staging area for users' job
files In practice, this should be the directory where users' home
directories are located (usually /user)  mapreduce.cluster.temp.dir
${hadoop.tmp.dir}/mapred/tempA shared directory for temporary files.
I am aware that mapreduce.cluster.local.dir can be multivalued and i can
exlicitly set this property but i was wondering that it would be even
better if i can set multiple values in hadoop.tmp.dir property. Also,
is mapreduce.cluster.temp.dir
property multivalued or single valued?

-- 
Thanks & Regards,
Anil Gupta

Mime
View raw message