hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shumin Wu <shumin...@gmail.com>
Subject Re: HDFS Namenode Format Question.
Date Fri, 14 Sep 2012 16:12:14 GMT
It looks like your hadoop picked up another configuration directory (most
likely the default one) other than the one you've customized. You may print
the path to your hadoop conf when start the namenode, and check which
hadoop conf it reads from.

-Shumin

On Fri, Sep 14, 2012 at 8:01 AM, Jason Huang <jason.huang@icare.com> wrote:

> Hello,
>
> I am trying to set up Hadoop 1.0.3 in my Macbook Pro in a
> pseudo-distributed mode.
>
> After download / install / setup config files I ran the following
> namenode format command as suggested in the user guide:
>
> $bin/hadoop namenode -format
>
> Here is the output:
> ************************************************************/
> 12/09/14 10:46:42 INFO util.GSet: VM type       = 32-bit
> 12/09/14 10:46:42 INFO util.GSet: 2% max memory = 39.6925 MB
> 12/09/14 10:46:42 INFO util.GSet: capacity      = 2^23 = 8388608 entries
> 12/09/14 10:46:42 INFO util.GSet: recommended=8388608, actual=8388608
> 12/09/14 10:46:42 INFO namenode.FSNamesystem: fsOwner=jasonhuang
> 12/09/14 10:46:42 INFO namenode.FSNamesystem: supergroup=supergroup
> 12/09/14 10:46:42 INFO namenode.FSNamesystem: isPermissionEnabled=true
> 12/09/14 10:46:42 INFO namenode.FSNamesystem:
> dfs.block.invalidate.limit=100
> 12/09/14 10:46:42 INFO namenode.FSNamesystem:
> isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s),
> accessTokenLifetime=0 min(s)
> 12/09/14 10:46:42 INFO namenode.NameNode: Caching file names occuring
> more than 10 times
> 12/09/14 10:46:42 INFO common.Storage: Image file of size 116 saved in
> 0 seconds.
> 12/09/14 10:46:42 INFO common.Storage: Storage directory
> /tmp/hadoop-jasonhuang/dfs/name has been successfully formatted.
> 12/09/14 10:46:42 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
>
> It appears that the storage directory is /tmp/hadoop-jasonhuang/dfs/name
>
> However, in my config file I've assigned a different directory (see
> hdfs-site.xml below):
> <configuration>
>   <property>
>      <name>dfs.replication</name>
>      <value>1</value>
>   </property>
>   <property>
>      <name>dfs.namenode.name.dir</name>
>      <value>/Users/jasonhuang/hdfs/name</value>
>   </property>
>   <property>
>      <name>dfs.datanode.data.dir</name>
>      <value>/Users/jasonhuang/hdfs/data</value>
>   </property>
>
> Does anyone know why the hdfs-site.xml might not be respected?
>
> Also, after formatting the name node, I did a search for the fsimage
> file in my local file directories (from root dir) and here is what I
> found:
> $ sudo find / -name fsimage
> /private/tmp/hadoop-jasonhuang/dfs/name/current/fsimage
> /private/tmp/hadoop-jasonhuang/dfs/name/image/fsimage
>
> I don't understand why the name node format picked (and created) these
> two directories...
>
> Any thoughts?
>
> Thanks!
>
> Jason
>

Mime
View raw message