hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Bieniosek <mich...@powerset.com>
Subject Re: Formatting the namenode
Date Fri, 15 Jun 2007 19:36:27 GMT
Go through hadoop-default.xml and look at all the paths that install into
/tmp/hadoop-${user.name}.  You should probably override all of those values
in hadoop-site.xml.

-Michael

On 6/15/07 12:23 PM, "Avinash Lakshman" <alakshman@facebook.com> wrote:

> Dfs and mapred are the directories created inside the
> /tmp/hadoop-<user-name> directory.
> 
> Thanks
> A
> 
> -----Original Message-----
> From: Michael Bieniosek [mailto:michael@powerset.com]
> Sent: Friday, June 15, 2007 11:31 AM
> To: hadoop-user@lucene.apache.org; Phantom
> Subject: Re: Formatting the namenode
> 
> In hadoop-default.xml you should find:
> 
> <property>
>   <name>hadoop.tmp.dir</name>
>   <value>/tmp/hadoop-${user.name}</value>
>   <description>A base for other temporary directories.</description>
> </property>
> 
> <property>
>   <name>dfs.name.dir</name>
>   <value>${hadoop.tmp.dir}/dfs/name</value>
>   <description>Determines where on the local filesystem the DFS name
> node
>       should store the name table.  If this is a comma-delimited list
>       of directories then the name table is replicated in all of the
>       directories, for redundancy. </description>
> </property>
> 
> <property>
>   <name>dfs.data.dir</name>
>   <value>${hadoop.tmp.dir}/dfs/data</value>
>   <description>Determines where on the local filesystem an DFS data node
>   should store its blocks.  If this is a comma-delimited
>   list of directories, then data will be stored in all named
>   directories, typically on different devices.
>   Directories that do not exist are ignored.
>   </description>
> </property>
> 
> and some other properties that use the hadoop.tmp.dir prefix.  What
> subdirectories did hadoop create inside /tmp/hadoop-${user.name}?
> 
> -Michael
> 
> On 6/14/07 10:39 PM, "Phantom" <ghostwhoowalks@gmail.com> wrote:
> 
>> Hi
>> 
>> When I format my namenode it does format the directory specified under
>> dfs.name.dir. However there is a folder under /tmp called
> hadoop-alakshman.
>> What is this for ? Will all blocks be stored under /var/hadoop
> somewhere.
>>  Here is a copy of hadoop-site.xml
>> 
>> <?xml version="1.0"?>
>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> 
>> <!-- Put site-specific property overrides in this file. -->
>> 
>> <configuration>
>>         <property>
>>                 <name>dfs.name.dir</name>
>>                 <value>/var/hadoop</value>
>>         </property>
>> 
>>         <property>
>>                 <name>fs.default.name</name>
>>                 <value>hdfs://hadoop001.sf2p.com:9000</value>
>>         </property>
>> 
>>         <property>
>>                 <name>mapred.job.tracker</name>
>>                 <value>hadoop001.sf2p.com:50029</value>
>>         </property>
>> 
>>         <property>
>>                 <name>mapred.job.tracker.info.port</name>
>>                 <value>50030</value>
>>         </property>
>> 
>>         <property>
>>                 <name>mapred.child.java.opts</name>
>>                 <value>-Xmx1024M</value>
>>         </property>
>> 
>>         <property>
>>                 <name>mapred.min.split.size</name>
>>                 <value>65536</value>
>>         </property>
>> 
>>         <property>
>>                 <name>dfs.replication</name>
>>                 <value>3</value>
>>         </property>
>> 
>> </configuration>
> 


Mime
View raw message