hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arpit Gupta <ar...@hortonworks.com>
Subject Re: datanode has no storageID
Date Tue, 28 Aug 2012 17:42:57 GMT
> 2. how/where does the namenode stores the datanodes's storageids ?

When the datanode connects with the namenode for the first time it will register with the
namenode and during the registration the of the datanode the storage id for the datanode is
generated.

> 
> 4. can I format/reset the namenode to enable the datanode to reconnect ?

If you do not care about the existing data you can certainly do that. Make sure you stop all
processes, format the namenode, clean up the data directories on all your datanodes (dfs.data.dir
prop in hdfs-site.xml) and then start the namenode and the datanodes.

--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/

On Aug 28, 2012, at 9:13 AM, boazya@post.tau.ac.il wrote:

> Hi,
> hope it's not a newby question...
> I installed several versions of hadoop for testing,
> (0.20.203, 0.21.0, and 1.0.3)
> on various machines.
> now I am using 1.0.3 on all the machines,
> I face a problem that in some of the machhines, the datanode gets no
> storageID from the namenode.
> where it works, the datanode has the following lines in the log file:
> (and current/VERSION has a storageID=< some ID > )
> ---------------------------------------------------------------
> 2012-08-28 19:04:31,415 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration
= DatanodeRegistration(datanode-works.cs.tau.ac.il:50010, storageID=DS-996163017-<machines-ip>-50010-1342683478942,
infoPort=50075, ipcPort=50020)
> 2012-08-28 19:04:31,418 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting
asynchronous block report scan
> 2012-08-28 19:04:31,418 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(<machines-ip>:50010,
storageID=DS-996163017-<machines-ip>-50010-1342683478942, infoPort=50075, ipcPort=50020)In
DataNode.run, data = FSDataset{dirpath='/var/cache/hdfs/hadoop-data-node/current'}
> 2012-08-28 19:04:31,419 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> -------------------------------------------------------------------------------
> 
> where it doesn't work I have only the first line and it hangs.
> (and current/VERSION has a 'storageID=' empty value line )
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 2012-08-28 18:42:01,297 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration
= DatanodeRegistration(machinename.cs.tau.ac.il:50010, storageID=, infoPort=50075, ipcPort=50020)
> 2012-08-28 18:42:01,287 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1. any Ideas?
> 2. how/where does the namenode stores the datanodes's storageids ?
> 3. how can I get a new storageid for a datanode or it's old ID ?
> 4. can I format/reset the namenode to enable the datanode to reconnect ?
> 
> thanks!
> -
> Boaz Yarom
> CS System Team
> 03-640-8961 / 7637
> 
> 


Mime
View raw message