hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gerrit Jansen van Vuuren <gerrit...@googlemail.com>
Subject Re: Namenode Role
Date Thu, 18 Nov 2010 10:30:29 GMT

There is some development going on in both yahoo and facebook about making
the namenode HA, but so far there is nothing released that will do this.
So to answer your question: no, the namenode is a single point of failure
with no possibility of switching during runtime.

The only solution is to:
-> write output namenode metadata to two locations: localdisk, and a ntfs
-> you must always run the seconday/checkpoint namenode. if not the Namenode
will never merge its edits.log file into the on disk image file. (the
primary namenode only merges edits.log into the image under two conditions :
restart, or the secondary namenode requests a checkpoint)
-> make backups with a cronned script requesting checkpoints via the
namenode http api, and store these backups off rack even off site.

Using another namenode when the current namenode fails is then a restore
from one of the backups you've made or using one of the checkpoints made by
the secondary namenode. But I can't stress enough the fact that you need to
make as many backups as possible of your metadata, or else total data loss
will occur if you can't recover the metadata.

Hope this helps.


On Thu, Nov 18, 2010 at 3:20 AM, Ozcan ILIKHAN <ilikhan@cs.wisc.edu> wrote:

> Currently in my mini cluster I have one active and one backup NameNode.
> Whenever I need backup NameNode to be active/regular NameNode, I shutdown it
> and restart in active mode. As far as I understand from documentation and
> code, there is no way to switch from backup to active role at run time.
> Does anyone have a better idea of handling this situation?
> Thanks,
> Ozcan.

View raw message