hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vrodio...@carrieriq.com>
Subject RE: Region Server not starting
Date Fri, 08 Nov 2013 18:56:43 GMT
Cleaning up Zookeeper  store should resolve this. Its quick but not 100% safe. The right way
is to launch  Zk CLI and manually check and verify all HBase related nodes
By default, its under /HBASE_TMP_DIR/zookeeper or if you have specified it explicitly - check
hbase.zookeeper.property.dataDir in hbase-site.xml

Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

From: Suresh V S [Suresh.S@igate.com]
Sent: Friday, November 08, 2013 12:31 AM
To: user@hbase.apache.org
Subject: Region Server not starting

Dear all,

I had to change the IP address of a hbase region server in the cluster.  After effecting the
IP change, all services in cluster came up fine, but for the region server on this node.

Kindly advise how to solve this problem.

The log on the region server with the error portion is as below:

2013-11-08 13:41:13,780 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Waiting
for Large Compaction Thread to finish...
2013-11-08 13:41:13,780 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Waiting
for Small Compaction Thread to finish...
2013-11-08 13:41:13,795 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node
/hbase/rs/harry,60020,1383898273175 already deleted, and this is not a retry
2013-11-08 13:41:13,795 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: Failed deleting
my ephemeral node
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /hbase/rs/harry,60020,1383898273175
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:127)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:974)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:963)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1037)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:815)
        at java.lang.Thread.run(Thread.java:662)
2013-11-08 13:41:13,812 INFO org.apache.zookeeper.ZooKeeper: Session: 0x14231e9cc68000c closed
2013-11-08 13:41:13,812 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2013-11-08 13:41:13,812 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: stopping
server null; zookeeper connection closed.
2013-11-08 13:41:13,813 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: regionserver60020
2013-11-08 13:41:13,813 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook
starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@225f1ae9<mailto:fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@225f1ae9>


Information contained and transmitted by this e-mail is confidential and proprietary to iGATE
and its affiliates and is intended for use only by the recipient. If you are not the intended
recipient, you are hereby notified that any dissemination, distribution, copying or use of
this e-mail is strictly prohibited and you are requested to delete this e-mail immediately
and notify the originator or mailadmin@igate.com <mailto:mailadmin@igate.com>. iGATE
does not enter into any agreement with any party by e-mail. Any views expressed by an individual
do not necessarily reflect the view of iGATE. iGATE is not responsible for the consequences
of any actions taken on the basis of information provided, through this email. The contents
of an attachment to this e-mail may contain software viruses, which could damage your own
computer system. While iGATE has taken every reasonable precaution to minimise this risk,
we cannot accept liability for any damage which you sustain as a result of software viruses.
You should carry out your own virus checks before opening an attachment. To know more about
iGATE please visit www.igate.com <http://www.igate.com>.

Confidentiality Notice:  The information contained in this message, including any attachments
hereto, may be confidential and is intended to be read only by the individual or entity to
whom this message is addressed. If the reader of this message is not the intended recipient
or an agent or designee of the intended recipient, please note that any review, use, disclosure
or distribution of this message or its attachments, in any form, is strictly prohibited. 
If you have received this message in error, please immediately notify the sender and/or Notifications@carrieriq.com
and delete or destroy any copy of this message and its attachments.

View raw message