hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From boa...@post.tau.ac.il
Subject Re: datanode has no storageID
Date Thu, 30 Aug 2012 07:13:32 GMT
hi,
thanks for the reply,
I stopped all the processes,
moved the hadoop-data-node folders from all,
moved hadoop-name-node folder on the namenode machine
and formatted the namenode.
still some machines, the same as before has no storageID in  
current/VERSION file.
these machine are'nt shown in the web-GUI.
and this is the last line from the datanode log file:
2012-08-30 10:05:29,430 INFO  
org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration =  
DatanodeRegistration(datanode-machine-name.cs.tau.ac.il:50010,  
storageID=, infoPort=50075, ipcPort=50020)


any ideas ?

-- 
Boaz Yarom
CS System Team
03-640-8961 / 7637



Quoting Arpit Gupta <arpit@hortonworks.com>:

>> 2. how/where does the namenode stores the datanodes's storageids ?
>
> When the datanode connects with the namenode for the first time it  
> will register with the namenode and during the registration the of  
> the datanode the storage id for the datanode is generated.
>
>>
>> 4. can I format/reset the namenode to enable the datanode to reconnect ?
>
> If you do not care about the existing data you can certainly do  
> that. Make sure you stop all processes, format the namenode, clean  
> up the data directories on all your datanodes (dfs.data.dir prop in  
> hdfs-site.xml) and then start the namenode and the datanodes.
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Aug 28, 2012, at 9:13 AM, boazya@post.tau.ac.il wrote:
>
>> Hi,
>> hope it's not a newby question...
>> I installed several versions of hadoop for testing,
>> (0.20.203, 0.21.0, and 1.0.3)
>> on various machines.
>> now I am using 1.0.3 on all the machines,
>> I face a problem that in some of the machhines, the datanode gets no
>> storageID from the namenode.
>> where it works, the datanode has the following lines in the log file:
>> (and current/VERSION has a storageID=< some ID > )
>> ---------------------------------------------------------------
>> 2012-08-28 19:04:31,415 INFO  
>> org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration =  
>> DatanodeRegistration(datanode-works.cs.tau.ac.il:50010,  
>> storageID=DS-996163017-<machines-ip>-50010-1342683478942,  
>> infoPort=50075, ipcPort=50020)
>> 2012-08-28 19:04:31,418 INFO  
>> org.apache.hadoop.hdfs.server.datanode.DataNode: Starting  
>> asynchronous block report scan
>> 2012-08-28 19:04:31,418 INFO  
>> org.apache.hadoop.hdfs.server.datanode.DataNode:  
>> DatanodeRegistration(<machines-ip>:50010,  
>> storageID=DS-996163017-<machines-ip>-50010-1342683478942,  
>> infoPort=50075, ipcPort=50020)In DataNode.run, data =  
>> FSDataset{dirpath='/var/cache/hdfs/hadoop-data-node/current'}
>> 2012-08-28 19:04:31,419 INFO org.apache.hadoop.ipc.Server: IPC  
>> Server Responder: starting
>> -------------------------------------------------------------------------------
>>
>> where it doesn't work I have only the first line and it hangs.
>> (and current/VERSION has a 'storageID=' empty value line )
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 2012-08-28 18:42:01,297 INFO  
>> org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration =  
>> DatanodeRegistration(machinename.cs.tau.ac.il:50010, storageID=,  
>> infoPort=50075, ipcPort=50020)
>> 2012-08-28 18:42:01,287 INFO org.apache.hadoop.ipc.Server: Starting  
>> SocketReader
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 1. any Ideas?
>> 2. how/where does the namenode stores the datanodes's storageids ?
>> 3. how can I get a new storageid for a datanode or it's old ID ?
>> 4. can I format/reset the namenode to enable the datanode to reconnect ?
>>
>> thanks!
>> -
>> Boaz Yarom
>> CS System Team
>> 03-640-8961 / 7637
>>
>>




Mime
View raw message