hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From MShaik <mshai...@aol.com>
Subject Re: Need Help on Hadoop cluster Setup
Date Fri, 22 Mar 2013 15:18:00 GMT

Thank you, Tariq.
After chang the namesapceID on datanodes, all datanodes are started.


Thank you once again...!


-----Original Message-----
From: Mohammad Tariq <dontariq@gmail.com>
To: user <user@hadoop.apache.org>
Sent: Fri, Mar 22, 2013 8:29 pm
Subject: Re: Need Help on Hadoop cluster Setup


sorry for the typo in the second line of the 2nd point. the path will be "/dfs.data.dir/current/VERSION".


Warm Regards,
Tariq
https://mtariq.jux.com/

cloudfront.blogspot.com





On Fri, Mar 22, 2013 at 8:27 PM, Mohammad Tariq <dontariq@gmail.com> wrote:

have you reformatted the hdfs?if that is the case it was, i think, not proper.
were the nodes which you attached serving some other cluster earlier?your
logs show that you are facing problems because of mismatch in the IDs of
the NN and the IDs which DNs have. to overcome this problem you can 
follow these steps :


1 - Stop all teh DNs.
2 - Go to the directory which is serving as your dfs.data.dir. Inside this directiry
you'll find a subdirectory ". there will be a file named as "VERSION"  in this 
directory. in this file you can see the namespaceID(probably the second line).
change it to match the namespaceID which is there in "dfs.name.dir/current/VERSION"
file.
3 - restart the processes.


HTH





Warm Regards,
Tariq
https://mtariq.jux.com/

cloudfront.blogspot.com






On Fri, Mar 22, 2013 at 8:04 PM, MShaik <mshaik86@aol.com> wrote:

Hi,


DataNode is not started on all the nodes, as tasktracker is started on all the nodes.


please find the below datanode log, please let me know the solution.


2013-03-22 19:52:27,380 INFO org.apache.hadoop.ipc.RPC: Server at n1.hc.com/192.168.1.110:54310
not available yet, Zzzzz...
2013-03-22 19:52:29,386 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: n1.hc.com/192.168.1.110:54310.
Already tried 0 time(s).
2013-03-22 19:52:30,411 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: n1.hc.com/192.168.1.110:54310.
Already tried 1 time(s).
2013-03-22 19:52:31,416 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: n1.hc.com/192.168.1.110:54310.
Already tried 2 time(s).
2013-03-22 19:52:32,420 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: n1.hc.com/192.168.1.110:54310.
Already tried 3 time(s).
2013-03-22 19:52:33,426 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: n1.hc.com/192.168.1.110:54310.
Already tried 4 time(s).
2013-03-22 19:52:49,162 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException:
Incompatible namespaceIDs in /home/hduser/hadoopdata: namenode namespaceID = 2050588793; datanode
namespaceID = 503772406
	at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:232)
	at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:147)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:385)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)


2013-03-22 19:52:49,168 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at n4.hc.com/192.168.1.113
************************************************************/


Thank's



-----Original Message-----
From: Mohammad Tariq <dontariq@gmail.com>
To: user <user@hadoop.apache.org>
Sent: Fri, Mar 22, 2013 7:07 pm
Subject: Re: Need Help on Hadoop cluster Setup


Hello Munavvar,


      It depends on your configuration where your DNs and TTs will run. If you have configured
all your slaves to run both the processes then they should. If they are not running then there
is definitely some problem. Could you please check your DN logs once and see if you find anything
unusual there. And you have to copy the files across all the machines.


You can do one more thing just to cross check. Point your web browser to the HDFS web UI(master_machine:9000)
to do that.


Warm Regards,
Tariq
https://mtariq.jux.com/

cloudfront.blogspot.com





On Fri, Mar 22, 2013 at 6:44 PM, Munnavar Sk <mshaik86@aol.com> wrote:


Hi ,



 
I am new to Hadoop and I am fighting with this last 20days, somehowI got very good stuff on
Hadoop.
 
But, some question are roaming around me...I hope, I can getthe answers from your end...!
 
I was setup a cluster in distributed mode with 5 nodes. Ihave configured Namenode and DataNodes
and all datannodes are able to logingfrom namenode without password.
Hadoop and Java installed on same location in all the Nodes.After starting the cluster, I
was check every node using with "jps"command.
NameNode it was shows that all demonsrunning(NameNode,JobTracker,SecondryNameNode). 
Same process is I applied for Datanodes. But, Some nodesonly showing that TaskTracer running,
only one node shows that DataNode andTaskTracker runs perfectly.
My Question is that the configuration files are required tocopy all the nodes which is located
in $HADOOP_HOME/conf directory?
And why that DataNode is not running on remaining nodes?
 
Please clarify this doubts, so that I can able to moveahead... :)
 
Thank you,
M Shaik
--------------
 
 




 








 

Mime
View raw message