hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: what will happen if a backup name node folder becomes unaccessible?
Date Fri, 20 Aug 2010 21:27:11 GMT
Whee, lets try it out:

Start with both paths available. ... Starts fine.
Store some files. ... Works.
rm -r the second path. ... Ouch.
Store some more files. ... Still Works. [Cuz the SNN hasn't sent us
stuff back yet]
Wait for checkpoint to hit.
And ...
Boom!

2010-08-21 02:42:00,385 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log
from 127.0.0.1
2010-08-21 02:42:00,385 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 37 Total time for transactions(ms): 6Number of
transactions batched in Syncs: 0 Number of syncs: 26 SyncTimes(ms):
307 277
2010-08-21 02:42:00,439 FATAL
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Fatal Error : All
storage directories are inaccessible.
2010-08-21 02:42:00,440 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at localhost/127.0.0.1
************************************************************/

So yes, as Edward says - never let this happen!

On Sat, Aug 21, 2010 at 2:26 AM, jiang licht <licht_jiang@yahoo.com> wrote:
> Using nfs folder to back up dfs meta information as follows,
>
> <property>
>         <name>dfs.name.dir</name>
>         <value>/hadoop/dfs/name,/hadoop-backup/dfs/name</value>
>     </property>
>
> where /hadoop-backup is on a backup machine and mounted on the master node.
>
> I have a question: if somehow, the backup folder becomes unavailable, will it freeze
master node? That is, will write operation simply hang up on this condition on the master
node? Or will master node log the problem and continues to work?
>
> Thanks,
>
> Michael
>
>
>



-- 
Harsh J
www.harshj.com

Mime
View raw message