hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tao Xiao <xiaotao.cs....@gmail.com>
Subject Re: How to set "hadoop.tmp.dir" if I have multiple disks per node?
Date Mon, 16 Dec 2013 09:11:39 GMT
Thanks.

In order to spread I/O among multiple disks, should I assign a
comma-separated list of directories which are located on different disks to
"hadoop.tmp.dir"?

for example,

 <property>
      <name>hadoop.tmp.dir</name>

<value>/mnt/disk1/hadoop_tmp_dir,/mnt/disk2/hadoop_tmp_dir,/mnt/disk3/hadoop_tmp_dir</value>
 </property>


2013/12/16 Shekhar Sharma <shekhar2581@gmail.com>

> hadoop.tmp.dir is a directory created on local file system
> For example if you have set hadoop.tmp.dir property to
> /home/training/hadoop
>
> This directory will be created when you format the namenode by running
> the command
> hadoop namenode -format
>
> When you open this folder
>
>
> you will see two subfolders dfs and mapred.
>
> the /home/training/hadoop/mapred folder will be on HDFS also
>
> Hope this clears
> Regards,
> Som Shekhar Sharma
> +91-8197243810
>
>
> On Mon, Dec 16, 2013 at 1:42 PM, Dieter De Witte <drdwitte@gmail.com>
> wrote:
> > Hi,
> >
> > Make sure to also set mapred.local.dir to the same set of output
> > directories, this is were the intermediate key-value pairs are stored!
> >
> > Regards, Dieter
> >
> >
> > 2013/12/16 Tao Xiao <xiaotao.cs.nju@gmail.com>
> >>
> >> I have ten disks per node,and I don't know what value I should set to
> >> "hadoop.tmp.dir". Some said this property refers to a location in local
> disk
> >> while some other said it refers to a directory in HDFS. I'm confused,
> who
> >> can explain it ?
> >>
> >> I want to spread I/O since I have ten disks per node, so should I set a
> >> comma-separated list of directories (which are on different disks) to
> >> "hadoop.tmp.dir" ?
> >
> >
>

Mime
View raw message