hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cliff palmer <palmercl...@gmail.com>
Subject Re: Hadoop startup problem - directory name required
Date Mon, 23 Aug 2010 13:49:22 GMT
Thanks Harsh, but I am still not sure I understand what is going on.
The directory specified in the dfs.name.dir property,
"/var/lib/hadoop-0.20/dfsname", does exist and rights to that directory have
been granted to the OS user that is running the Hadoop startup script.
The directory mentioned in the error message is
"/var/lib/hadoop-0.20/cache/hadoop/dfs/name".
I can create this directory and that would (I assume) remove the error, but
I want to understand how the name is derived.  It's not a child of the
directory name specified in the dfs.name.dir property.
Thanks again!
Cliff


On Mon, Aug 23, 2010 at 9:21 AM, Harsh J <qwertymaniac@gmail.com> wrote:

> Its checking this directory cause your dfs.name.dir (hdfs-site.xml)
> has it in its list of dirs to write a copy to:
>
> <property>
>               <name>dfs.name.dir</name>
>               <value>/DFS/dfsname,/var/lib/hadoop-0.20/dfsname</value>
> <property>
>
> Remove it from this property if you don't need it. If you need
> multiple dirs, you should create the path and allocate proper
> permissions to it so that it may write to it. Its good to use a second
> path for backup purposes (most have this on the NFS). The path is so
> because your distribution of hadoop perhaps placed its files around
> that place and its up in the conf files as some form of default :)
>
> On Mon, Aug 23, 2010 at 6:45 PM, cliff palmer <palmercliff@gmail.com>
> wrote:
> > The namenode log for a Hadoop-0.20 installation contains this error
> message:
> > "/var/lib/hadoop-0.20/cache/hadoop/dfs/name in in an inconsistent state".
> > This directory does not exist and I would like to understand why this
> > particular directory name is required (not what the directory is used
> for,
> > but why this particular directory name).  The *-site.xml files are below
> (IP
> > addresses have been masked).
> > Thanks in advance for your help.
> > Cliff
> >
> > core-site.xml:
> >
> > <?xml version="1.0"?>
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >
> > <!-- Put site-specific property overrides in this file. -->
> >
> > <configuration>
> >        <property>
> >                <name>fs.default.name</name>
> >                <value>hdfs://xxx.xxx.xxx.xxx:8020</value>
> >                <final>true</final>
> >        </property>
> >        <property>
> >                <name>hadoop.tmp.dir</name>
> >                <value>/var/lib/hadoop-0.20/cache/hadoop</value>
> >        </property>
> > </configuration>
> > -------- end of core-site.xml -----------
> >
> > hdfs-site.xml:
> >
> > <?xml version="1.0"?>
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >
> > <!-- Put site-specific property overrides in this file. -->
> >
> > <configuration>
> >        <property>
> >                <name>dfs.replication</name>
> >                <value>3</value>
> >        </property>
> >        <property>
> >                <name>dfs.name.dir</name>
> >                <value>/DFS/dfsname,/var/lib/hadoop-0.20/dfsname</value>
> >        <property>
> >        </property>
> >                <name>dfs.data.dir</name>
> >                <value>/DFS1/dfsdata,/DFS2/dfsdata,/DFS3/dfsdata</value>
> >        </property>
> >        <property>
> >                <name>dfs.permissions</name>
> >                <value>false</value>
> >        </property>
> >        <property>
> >                <name>dfs.datanode.du.reserved</name>
> >                <value>20000000000</value>
> >        </property>
> >        <property>
> >                <name>dfs.namenode,plugins</name>
> >                <value>org.apache.hadoop.thriftfs.NamenodePlugin</value>
> >        </property>
> >        <property>
> >                <name>dfs.datanode.plugins</name>
> >                <value>org.apache.hadoop.thriftfs.DatanodePlugins</value>
> >        </property>
> >        <property>
> >                <name>dfs.thrift.address</name>
> >                <value>0.0.0.0:9090</value>
> >        </property>
> > </configuration>
> > -------- end of core-site.xml -----------
> >
> > mapred-site.xml:
> >
> > <?xml version="1.0"?>
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >
> > <!-- Put site-specific property overrides in this file. -->
> >
> > <configuration>
> >        <property>
> >                <name>mapred.job.tracker</name>
> >                <value>xxx.xxx.xxx.xxx:8020</value>
> >        </property>
> >        <property>
> >                <name>jobtracker.thrift.address</name>
> >                <value>0.0.0:9290</value>
> >        </property>
> >        <property>
> >                <name>mapred.jobtracker.plugins</name>
> >
> > <value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>
> >        </property>
> > </configuration>
> > -------- end of core-site.xml -----------
> >
>
>
>
> --
> Harsh J
> www.harshj.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message