hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lohit <lohit...@yahoo.com>
Subject Re: NameNode does not come out of Safemode automatically in Hadoop-0.17.2
Date Fri, 14 Nov 2008 05:34:00 GMT
Namenode does not come out of safemode until it gets confirmation from datanodes about the
blocks it has. Namenode has a view of filesystem and its blocks, it expects those blocks to
be reported by datanode until which it decides that the filesystem is not ready to use yet.
You can exit the namenode from safemode and do 'dfs fsck / -files -blocks -locations' which
will tell you the missing blocks and their locations where it expects from. Check if those
nodes are up and running an they have those blocks. 

----- Original Message ----
From: Pratyush Banerjee <pratyushbanerjee@aol.com>
To: core-user@hadoop.apache.org
Sent: Thursday, November 13, 2008 8:39:20 PM
Subject: NameNode does not come out of Safemode automatically in Hadoop-0.17.2

Hi All,

We have been using hadoop-0.17.2 for some time now and we just had a case of namenode crash
due to disk being full.
In order to get the namenode up again with minimal loss of data, we had to manually edit the
edits file in a Hex editor and restart the namenode.

However after restarting, the namenode went to the safe mode (as expected), but it has been
hours since it is like that, and it has not yet come out of the  safemode.
We can obviously force it to come out but should it not come out automatically ?
Even after 12 hours of remaining in safemode the ratio of reported block size is still stuck
at  0.9768.

Running fsck on / in the hdfs does report about some corrupt files.

What is the  issue which is blocking namenode form coming out of safemode ? If we have to
do it manually (hadoop dfsadmin -safemode leave) then what procedure do we follow in the process
to ensure data safety ?

thanks and regards,


View raw message