hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: HDFS Namenode Format Question.
Date Fri, 14 Sep 2012 17:36:41 GMT
If you are using 1.0.3, then the config names are wrong. You need
dfs.name.dir and dfs.data.dir instead. Those configs you have are for
2.x based releases.

Also, I'd make that look like ${user.home}/hdfs/name, etc. for a slightly more
portable/templatey config :)

On Fri, Sep 14, 2012 at 8:31 PM, Jason Huang <jason.huang@icare.com> wrote:
> Hello,
>
> I am trying to set up Hadoop 1.0.3 in my Macbook Pro in a
> pseudo-distributed mode.
>
> After download / install / setup config files I ran the following
> namenode format command as suggested in the user guide:
>
> $bin/hadoop namenode -format
>
> Here is the output:
> ************************************************************/
> 12/09/14 10:46:42 INFO util.GSet: VM type       = 32-bit
> 12/09/14 10:46:42 INFO util.GSet: 2% max memory = 39.6925 MB
> 12/09/14 10:46:42 INFO util.GSet: capacity      = 2^23 = 8388608 entries
> 12/09/14 10:46:42 INFO util.GSet: recommended=8388608, actual=8388608
> 12/09/14 10:46:42 INFO namenode.FSNamesystem: fsOwner=jasonhuang
> 12/09/14 10:46:42 INFO namenode.FSNamesystem: supergroup=supergroup
> 12/09/14 10:46:42 INFO namenode.FSNamesystem: isPermissionEnabled=true
> 12/09/14 10:46:42 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
> 12/09/14 10:46:42 INFO namenode.FSNamesystem:
> isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s),
> accessTokenLifetime=0 min(s)
> 12/09/14 10:46:42 INFO namenode.NameNode: Caching file names occuring
> more than 10 times
> 12/09/14 10:46:42 INFO common.Storage: Image file of size 116 saved in
> 0 seconds.
> 12/09/14 10:46:42 INFO common.Storage: Storage directory
> /tmp/hadoop-jasonhuang/dfs/name has been successfully formatted.
> 12/09/14 10:46:42 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
>
> It appears that the storage directory is /tmp/hadoop-jasonhuang/dfs/name
>
> However, in my config file I've assigned a different directory (see
> hdfs-site.xml below):
> <configuration>
>   <property>
>      <name>dfs.replication</name>
>      <value>1</value>
>   </property>
>   <property>
>      <name>dfs.namenode.name.dir</name>
>      <value>/Users/jasonhuang/hdfs/name</value>
>   </property>
>   <property>
>      <name>dfs.datanode.data.dir</name>
>      <value>/Users/jasonhuang/hdfs/data</value>
>   </property>
>
> Does anyone know why the hdfs-site.xml might not be respected?
>
> Also, after formatting the name node, I did a search for the fsimage
> file in my local file directories (from root dir) and here is what I
> found:
> $ sudo find / -name fsimage
> /private/tmp/hadoop-jasonhuang/dfs/name/current/fsimage
> /private/tmp/hadoop-jasonhuang/dfs/name/image/fsimage
>
> I don't understand why the name node format picked (and created) these
> two directories...
>
> Any thoughts?
>
> Thanks!
>
> Jason



-- 
Harsh J

Mime
View raw message