hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: how to restart a hbase cluster
Date Tue, 19 Jul 2011 07:10:37 GMT
On Tue, Jul 19, 2011 at 12:02 AM, Weihua JIANG <weihua.jiang@gmail.com> wrote:
> It seems stop-hbase.sh only stops master/backup masters and zookeepers.
>

Usually it sends a signal to the master that then sets a flag in
zookeeper.  When regionservers see this flag, they start to close down
user-space regions.  When all user-space regions have been closed,
they the server will close catalog regions.  When a regionserver is
carrying no regions, it shuts itself down.

The master waits until all regionservers are down.  It then will go down itself.

If you have set hbase to manage zookeeper, the last thing done on the
way out is shutdown the zk ensemble.

This is how it is supposed to work.


> So, according to my understanding, region servers shall shutdown
> itself since it can't find either master or zookeeper.
>

Hmm  Don't they keep retrying?


> But, I made a recent experimentation on our hbase cluster. After 2
> days of mater/zookeeper shutdown, the region servers are still alive.

That doesn't seem correct.  Did the cluster come up cleanly?  Or did
the master go down before regionservers came up?

> I am not sure whether it is the problem in hbase release or our own
> problem since our version is a heavy patched one.
>
> Then, can I perform hbase cluster in following way?
> 1. stop master
> 2. stop master backups
> 3. stop zookeepers
> 4. stop region servers
>
> The only difference is step #4. If I manually stop down RS, will it
> affect data integrity? If not, then I can safely performed the steps
> to shutdown the cluster.
>

If a regionserver crashes down rather than shutdown cleanly, it will
leave its wal logs around.  The master will notice them and replay
them.  So try not to crash out your regionservers. ./bin/stop-hbase.sh
should put the regionservers all down cleanly.

If you do ./bin/hbase-daemon.sh stop regionserver, that'll send the
process a signal.  It'll run its shutdown signal handler.  I think
this will bring on a clean shutdown.  See the code to be sure.

if  clean shutdown, data should be preserved.   Even if its not a
clean shutdown, as long as the log splitting is allowed complete,
there should be no data loss even if server is crashed down.

St.Ack

Mime
View raw message