hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Nelson <had...@enigmasecurity.com>
Subject Re: Datanode not found
Date Tue, 08 Apr 2008 05:05:52 GMT
I had been getting this exact same problem on my cluster. The solution I 
*finally* came up with (after 1-2 weeks of messing) was to manually 
insert a unique datanode ID into the datadir/current/VERSION file. If 
your problem is the same as mine your VERSION file may look like:

#Fri Apr 04 00:43:15 CDT 2008
namespaceID=1031383270
storageID=
cTime=0
storageType=DATA_NODE
layoutVersion=-11

Or something like that with a blank storageID field. The normal value 
for this field is DS-(random ~10 digit number)-IP-dataport-(random ~13 
digit number). I don't think this value really matters *what* it is so 
long as it's unique for each datanode (it could just be an IP address 
since usually in production you only run 1 datanode per machine). After 
I filled the storageID field in my datanodes started connecting to my 
namenode properly. I plan to try and debug this and submit a patch but 
I'm having problems getting my development environment to pass all the 
junit tests on vanilla code. Please let me know if this hack works for you.

Regards,
Tim Nelson

Rui Shi wrote:

>Hi,
>
>I am configuring a cluster using 0.16.2. But when the cluster starts, it can not find
any live datanode. I check the logs of one datanodes, which seems hanging in startup:
>2008-04-05 17:48:17,967 INFO org.apache.hadoop.dfs.DataNode: Registered FSDatasetStatusMBean
>2008-04-05 17:48:17,968 INFO org.apache.hadoop.dfs.DataNode: Opened server at 50010
>2008-04-05 17:48:17,971 INFO org.apache.hadoop.dfs.DataNode: Balancing bandwith is 1048576
bytes/s
>2008-04-05 17:48:18,465 INFO org.mortbay.util.Credential: Checking Resource aliases
>2008-04-05 17:48:18,653 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4
>2008-04-05 17:48:19,784 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@112e7f7
>2008-04-05 17:48:19,849 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/]
>2008-04-05 17:48:19,850 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs]
>2008-04-05 17:48:19,851 INFO org.mortbay.util.Container: Started HttpContext[/static,/static]
>2008-04-05 17:48:19,854 INFO org.mortbay.http.SocketListener: Started SocketListener on
0.0.0.0:50075
>2008-04-05 17:48:19,855 INFO org.mortbay.util.Container: Started org.mortbay.jetty.Server@13f991Does
anyone know how to resolve it?
>
>Thanks,
>
>Rui
>
>
>
>      ____________________________________________________________________________________
>You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access,
No Cost.  
>http://tc.deals.yahoo.com/tc/blockbuster/text5.com
>  
>


Mime
View raw message