hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zander1013 <zander1...@gmail.com>
Subject Re: datanode not being started
Date Tue, 17 Feb 2009 03:49:29 GMT

hi,

i am not seeing the DataNode run either. but i am seeing an extra process
TaskTracker run.

here is what hapens when i start the cluster run jps and stop the cluster...

hadoop@node0:/usr/local/hadoop$ bin/start-all.sh
starting namenode, logging to
/usr/local/hadoop/bin/../logs/hadoop-hadoop-namenode-node0.out
node0.local: starting datanode, logging to
/usr/local/hadoop/bin/../logs/hadoop-hadoop-datanode-node0.out
node1.local: starting datanode, logging to
/usr/local/hadoop/bin/../logs/hadoop-hadoop-datanode-node1.out
node0.local: starting secondarynamenode, logging to
/usr/local/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-node0.out
starting jobtracker, logging to
/usr/local/hadoop/bin/../logs/hadoop-hadoop-jobtracker-node0.out
node0.local: starting tasktracker, logging to
/usr/local/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node0.out
node1.local: starting tasktracker, logging to
/usr/local/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node1.out
hadoop@node0:/usr/local/hadoop$ jps
13353 TaskTracker
13126 SecondaryNameNode
12846 NameNode
13455 Jps
13232 JobTracker
hadoop@node0:/usr/local/hadoop$ bin/stop-all.sh
stopping jobtracker
node0.local: stopping tasktracker
node1.local: stopping tasktracker
stopping namenode
node0.local: no datanode to stop
node1.local: no datanode to stop
node0.local: stopping secondarynamenode
hadoop@node0:/usr/local/hadoop$ 

here is the tail of the log file for the session above...
************************************************************/
2009-02-16 19:35:13,999 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG:   host = node1/127.0.1.1
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.19.0
STARTUP_MSG:   build =
https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19 -r 713890;
compiled by 'ndaley' on Fri Nov 14 03:12:29 UTC 2008
************************************************************/
2009-02-16 19:35:18,999 ERROR
org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException:
Incompatible namespaceIDs in
/usr/local/hadoop-datastore/hadoop-hadoop/dfs/data: namenode namespaceID =
1050914495; datanode namespaceID = 722953254
    at
org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:233)
    at
org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:148)
    at
org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:287)
    at
org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:205)
    at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1199)
    at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1154)
    at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1162)
    at
org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1284)

2009-02-16 19:35:19,000 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at node1/127.0.1.1
************************************************************/

i have not seen DataNode run yet. i have only started and stopped the
cluster a couple of times.

i tried to reformat datanode and namenode with bin/hadoop datanode -format
and bin/hadoop namenode -format from /usr/local/hadoop dir.

please advise

zander



Mithila Nagendra wrote:
> 
> Hey Sandy
> I had a similar problem with Hadoop. All I did was I stopped all the
> daemons
> using stop-all.sh. Then formatted the namenode again using hadoop namenode
> -format. After this I went on to restarting everything by using
> start-all.sh
> 
> I hope you dont have much data on the datanode, reformatting it would
> erase
> everything out.
> 
> Hope this helps!
> Mithila
> 
> 
> 
> On Sat, Feb 14, 2009 at 2:39 AM, james warren <james@rockyou.com> wrote:
> 
>> Sandy -
>>
>> I suggest you take a look into your NameNode and DataNode logs.  From the
>> information posted, these likely would be at
>>
>>
>> /Users/hadoop/hadoop-0.18.2/bin/../logs/hadoop-hadoop-namenode-loteria.cs.tamu.edu.log
>>
>> /Users/hadoop/hadoop-0.18.2/bin/../logs/hadoop-hadoop-jobtracker-loteria.cs.tamu.edu.log
>>
>> If the cause isn't obvious from what you see there, could you please post
>> the last few lines from each log?
>>
>> -jw
>>
>> On Fri, Feb 13, 2009 at 3:28 PM, Sandy <snickerdoodle08@gmail.com> wrote:
>>
>> > Hello,
>> >
>> > I would really appreciate any help I can get on this! I've suddenly ran
>> > into
>> > a very strange error.
>> >
>> > when I do:
>> > bin/start-all
>> > I get:
>> > hadoop$ bin/start-all.sh
>> > starting namenode, logging to
>> >
>> >
>> /Users/hadoop/hadoop-0.18.2/bin/../logs/hadoop-hadoop-namenode-loteria.cs.tamu.edu.out
>> > starting jobtracker, logging to
>> >
>> >
>> /Users/hadoop/hadoop-0.18.2/bin/../logs/hadoop-hadoop-jobtracker-loteria.cs.tamu.edu.out
>> >
>> > No datanode, secondary namenode or jobtracker are being started.
>> >
>> > When I try to upload anything on the dfs, I get a "node in safemode"
>> error
>> > (even after waiting 5 minutes), presumably because it's trying to reach
>> a
>> > datanode that does not exist.  The same "safemode" error occurs when I
>> try
>> > to run jobs.
>> >
>> > I have tried bin/stop-all and then bin/start-all again. I get the same
>> > problem!
>> >
>> > This is incredibly strange, since I was previously able to start and
>> run
>> > jobs without any issue using this version on this machine. I am running
>> > jobs
>> > on a single Mac Pro running OS X 10.5
>> >
>> > I have tried updating to hadoop-0.19.0, and I get the same problem. I
>> have
>> > even tried this using previous versions, and I'm getting the same
>> problem!
>> >
>> > Anyone have any idea why this suddenly could be happening? What am I
>> doing
>> > wrong?
>> >
>> > For convenience, I'm including portions of both conf/hadoop-env.sh and
>> > conf/hadoop-site.xml:
>> >
>> > --- hadoop-env.sh ---
>> >  # Set Hadoop-specific environment variables here.
>> >
>> > # The only required environment variable is JAVA_HOME.  All others are
>> > # optional.  When running a distributed configuration it is best to
>> > # set JAVA_HOME in this file, so that it is correctly defined on
>> > # remote nodes.
>> >
>> > # The java implementation to use.  Required.
>> >  export
>> >
>> JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
>> >
>> > # Extra Java CLASSPATH elements.  Optional.
>> > # export HADOOP_CLASSPATH=
>> >
>> > # The maximum amount of heap to use, in MB. Default is 1000.
>> >  export HADOOP_HEAPSIZE=3000
>> > ...
>> > --- hadoop-site.xml ---
>> > <configuration>
>> >
>> > <property>
>> >  <name>hadoop.tmp.dir</name>
>> >  <value>/Users/hadoop/hadoop-0.18.2/hadoop-${user.name}</value>
>> >  <description>A base for other temporary directories.</description>
>> > </property>
>> >
>> > <property>
>> >  <name>fs.default.name</name>
>> >  <value>hdfs://localhost:9000</value>
>> >  <description>The name of the default file system.  A URI whose
>> >  scheme and authority determine the FileSystem implementation.  The
>> >  uri's scheme determines the config property (fs.SCHEME.impl) naming
>> >  the FileSystem implementation class.  The uri's authority is used to
>> >  determine the host, port, etc. for a filesystem.</description>
>> > </property>
>> >
>> > <property>
>> >  <name>mapred.job.tracker</name>
>> >  <value>localhost:9001</value>
>> >  <description>The host and port that the MapReduce job tracker runs
>> >  at.  If "local", then jobs are run in-process as a single map
>> >  and reduce task.
>> >  </description>
>> > </property>
>> >
>> > <property>
>> > <name>mapred.tasktracker.tasks.maximum</name>
>> > <value>1</value>
>> > <description>The maximum number of tasks that will be run
>> simultaneously
>> by
>> > a
>> > a task tracker
>> > </description>
>> > </property>
>> > ...
>> >
>>
> 
> 

-- 
View this message in context: http://www.nabble.com/datanode-not-being-started-tp22006929p22049288.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.


Mime
View raw message