hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: Need Help on Hadoop cluster Setup
Date Fri, 22 Mar 2013 14:58:38 GMT
sorry for the typo in the second line of the 2nd point. the path will be
"/dfs.data.dir/current/VERSION".

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Fri, Mar 22, 2013 at 8:27 PM, Mohammad Tariq <dontariq@gmail.com> wrote:

> have you reformatted the hdfs?if that is the case it was, i think, not
> proper.
> were the nodes which you attached serving some other cluster earlier?your
> logs show that you are facing problems because of mismatch in the IDs of
> the NN and the IDs which DNs have. to overcome this problem you can
> follow these steps :
>
> 1 - Stop all teh DNs.
> 2 - Go to the directory which is serving as your dfs.data.dir. Inside this
> directiry
> you'll find a subdirectory ". there will be a file named as "VERSION"  in
> this
> directory. in this file you can see the namespaceID(probably the second
> line).
> change it to match the namespaceID which is there in
> "dfs.name.dir/current/VERSION"
> file.
> 3 - restart the processes.
>
> HTH
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Fri, Mar 22, 2013 at 8:04 PM, MShaik <mshaik86@aol.com> wrote:
>
>>  Hi,
>>
>>  DataNode is not started on all the nodes, as tasktracker is started on
>> all the nodes.
>>
>>  please find the below datanode log, please let me know the solution.
>>
>>  2013-03-22 19:52:27,380 INFO org.apache.hadoop.ipc.RPC: Server at
>> n1.hc.com/192.168.1.110:54310 not available yet, Zzzzz...
>> 2013-03-22 19:52:29,386 INFO org.apache.hadoop.ipc.Client: Retrying
>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 0
>> time(s).
>> 2013-03-22 19:52:30,411 INFO org.apache.hadoop.ipc.Client: Retrying
>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 1
>> time(s).
>> 2013-03-22 19:52:31,416 INFO org.apache.hadoop.ipc.Client: Retrying
>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 2
>> time(s).
>> 2013-03-22 19:52:32,420 INFO org.apache.hadoop.ipc.Client: Retrying
>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 3
>> time(s).
>> 2013-03-22 19:52:33,426 INFO org.apache.hadoop.ipc.Client: Retrying
>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 4
>> time(s).
>> 2013-03-22 19:52:49,162 ERROR
>> org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException:
>> Incompatible namespaceIDs in /home/hduser/hadoopdata: namenode namespaceID
>> = 2050588793; datanode namespaceID = 503772406
>>  at
>> org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:232)
>>  at
>> org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:147)
>>  at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:385)
>>  at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
>>  at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
>>  at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
>>  at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
>>  at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
>>  at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
>>
>>  2013-03-22 19:52:49,168 INFO
>> org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down DataNode at n4.hc.com/192.168.1.113
>> ************************************************************/
>>
>>
>> Thank's
>>
>> -----Original Message-----
>> From: Mohammad Tariq <dontariq@gmail.com>
>> To: user <user@hadoop.apache.org>
>> Sent: Fri, Mar 22, 2013 7:07 pm
>> Subject: Re: Need Help on Hadoop cluster Setup
>>
>>  Hello Munavvar,
>>
>>        It depends on your configuration where your DNs and TTs will run.
>> If you have configured all your slaves to run both the processes then they
>> should. If they are not running then there is definitely some problem.
>> Could you please check your DN logs once and see if you find anything
>> unusual there. And you have to copy the files across all the machines.
>>
>>  You can do one more thing just to cross check. Point your web browser
>> to the HDFS web UI(master_machine:9000) to do that.
>>
>>  Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>>  cloudfront.blogspot.com
>>
>>
>> On Fri, Mar 22, 2013 at 6:44 PM, Munnavar Sk <mshaik86@aol.com> wrote:
>>
>>>
>>> Hi ,
>>>
>>> I am new to Hadoop and I am fighting with this last 20days, somehow I
>>> got very good stuff on Hadoop.
>>>
>>> But, some question are roaming around me...I hope, I can get the answers
>>> from your end...!
>>>
>>> I was setup a cluster in distributed mode with 5 nodes. I have
>>> configured Namenode and DataNodes and all datannodes are able to loging
>>> from namenode without password.
>>> Hadoop and Java installed on same location in all the Nodes. After
>>> starting the cluster, I was check every node using with "jps" command.
>>> NameNode it was shows that all demons
>>> running(NameNode,JobTracker,SecondryNameNode).
>>> Same process is I applied for Datanodes. But, Some nodes only showing
>>> that TaskTracer running, only one node shows that DataNode and TaskTracker
>>> runs perfectly.
>>> My Question is that the configuration files are required to copy all the
>>> nodes which is located in $HADOOP_HOME/conf directory?
>>> And why that DataNode is not running on remaining nodes?
>>>
>>> Please clarify this doubts, so that I can able to move ahead... :)
>>>
>>> Thank you,
>>> M Shaik
>>> --------------
>>>
>>
>>
>

Mime
View raw message