hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: Need Help on Hadoop cluster Setup
Date Fri, 22 Mar 2013 15:45:46 GMT
you are welcome.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Fri, Mar 22, 2013 at 8:48 PM, MShaik <mshaik86@aol.com> wrote:

>
> Thank you, Tariq.
> After chang the namesapceID on datanodes, all datanodes are started.
>
>  Thank you once again...!
>
> -----Original Message-----
> From: Mohammad Tariq <dontariq@gmail.com>
> To: user <user@hadoop.apache.org>
> Sent: Fri, Mar 22, 2013 8:29 pm
> Subject: Re: Need Help on Hadoop cluster Setup
>
>  sorry for the typo in the second line of the 2nd point. the path will be
> "/dfs.data.dir/current/VERSION".
>
>  Warm Regards,
> Tariq
> https://mtariq.jux.com/
>  cloudfront.blogspot.com
>
>
> On Fri, Mar 22, 2013 at 8:27 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>
>> have you reformatted the hdfs?if that is the case it was, i think, not
>> proper.
>> were the nodes which you attached serving some other cluster earlier?your
>> logs show that you are facing problems because of mismatch in the IDs of
>> the NN and the IDs which DNs have. to overcome this problem you can
>> follow these steps :
>>
>>  1 - Stop all teh DNs.
>> 2 - Go to the directory which is serving as your dfs.data.dir. Inside
>> this directiry
>> you'll find a subdirectory ". there will be a file named as "VERSION"  in
>> this
>> directory. in this file you can see the namespaceID(probably the second
>> line).
>>  change it to match the namespaceID which is there in
>> "dfs.name.dir/current/VERSION"
>> file.
>> 3 - restart the processes.
>>
>>  HTH
>>
>>
>>  Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>>  cloudfront.blogspot.com
>>
>>
>>   On Fri, Mar 22, 2013 at 8:04 PM, MShaik <mshaik86@aol.com> wrote:
>>
>>>  Hi,
>>>
>>>  DataNode is not started on all the nodes, as tasktracker is started on
>>> all the nodes.
>>>
>>>  please find the below datanode log, please let me know the solution.
>>>
>>>  2013-03-22 19:52:27,380 INFO org.apache.hadoop.ipc.RPC: Server at
>>> n1.hc.com/192.168.1.110:54310 not available yet, Zzzzz...
>>> 2013-03-22 19:52:29,386 INFO org.apache.hadoop.ipc.Client: Retrying
>>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 0
>>> time(s).
>>> 2013-03-22 19:52:30,411 INFO org.apache.hadoop.ipc.Client: Retrying
>>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 1
>>> time(s).
>>> 2013-03-22 19:52:31,416 INFO org.apache.hadoop.ipc.Client: Retrying
>>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 2
>>> time(s).
>>> 2013-03-22 19:52:32,420 INFO org.apache.hadoop.ipc.Client: Retrying
>>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 3
>>> time(s).
>>> 2013-03-22 19:52:33,426 INFO org.apache.hadoop.ipc.Client: Retrying
>>> connect to server: n1.hc.com/192.168.1.110:54310. Already tried 4
>>> time(s).
>>> 2013-03-22 19:52:49,162 ERROR
>>> org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException:
>>> Incompatible namespaceIDs in /home/hduser/hadoopdata: namenode namespaceID
>>> = 2050588793; datanode namespaceID = 503772406
>>>  at
>>> org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:232)
>>>  at
>>> org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:147)
>>>  at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:385)
>>>  at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
>>>  at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
>>>  at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
>>>  at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
>>>  at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
>>>  at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
>>>
>>>  2013-03-22 19:52:49,168 INFO
>>> org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
>>> /************************************************************
>>> SHUTDOWN_MSG: Shutting down DataNode at n4.hc.com/192.168.1.113
>>> ************************************************************/
>>>
>>>
>>> Thank's
>>>
>>> -----Original Message-----
>>> From: Mohammad Tariq <dontariq@gmail.com>
>>> To: user <user@hadoop.apache.org>
>>> Sent: Fri, Mar 22, 2013 7:07 pm
>>> Subject: Re: Need Help on Hadoop cluster Setup
>>>
>>>  Hello Munavvar,
>>>
>>>        It depends on your configuration where your DNs and TTs will
>>> run. If you have configured all your slaves to run both the processes then
>>> they should. If they are not running then there is definitely some problem.
>>> Could you please check your DN logs once and see if you find anything
>>> unusual there. And you have to copy the files across all the machines.
>>>
>>>  You can do one more thing just to cross check. Point your web browser
>>> to the HDFS web UI(master_machine:9000) to do that.
>>>
>>>  Warm Regards,
>>> Tariq
>>> https://mtariq.jux.com/
>>>  cloudfront.blogspot.com
>>>
>>>
>>> On Fri, Mar 22, 2013 at 6:44 PM, Munnavar Sk <mshaik86@aol.com> wrote:
>>>
>>>>
>>>> Hi ,
>>>>
>>>> I am new to Hadoop and I am fighting with this last 20days, somehow I
>>>> got very good stuff on Hadoop.
>>>>
>>>> But, some question are roaming around me...I hope, I can get the
>>>> answers from your end...!
>>>>
>>>> I was setup a cluster in distributed mode with 5 nodes. I have
>>>> configured Namenode and DataNodes and all datannodes are able to loging
>>>> from namenode without password.
>>>> Hadoop and Java installed on same location in all the Nodes. After
>>>> starting the cluster, I was check every node using with "jps" command.
>>>> NameNode it was shows that all demons
>>>> running(NameNode,JobTracker,SecondryNameNode).
>>>> Same process is I applied for Datanodes. But, Some nodes only showing
>>>> that TaskTracer running, only one node shows that DataNode and TaskTracker
>>>> runs perfectly.
>>>> My Question is that the configuration files are required to copy all
>>>> the nodes which is located in $HADOOP_HOME/conf directory?
>>>> And why that DataNode is not running on remaining nodes?
>>>>
>>>> Please clarify this doubts, so that I can able to move ahead... :)
>>>>
>>>> Thank you,
>>>> M Shaik
>>>> --------------
>>>>
>>>
>>>
>>
>

Mime
View raw message