hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-6325) [replication] Race in ReplicationSource.init can initiate a failover even if the node is alive
Date Tue, 03 Jul 2012 23:30:33 GMT
Jean-Daniel Cryans created HBASE-6325:
-----------------------------------------

             Summary: [replication] Race in ReplicationSource.init can initiate a failover
even if the node is alive
                 Key: HBASE-6325
                 URL: https://issues.apache.org/jira/browse/HBASE-6325
             Project: HBase
          Issue Type: Bug
    Affects Versions: 0.94.0, 0.92.1, 0.90.6
            Reporter: Jean-Daniel Cryans
             Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.2


Yet another bug found during the leap second madness, it's possible to miss the registration
of new region servers so that in ReplicationSource.init we start the failover of a live and
replicating region server. I don't think there's data loss but the RS that's being failed
over will die on:

{noformat}
2012-07-01 06:25:15,604 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING
region server sv4r23s48,10304,1341112194623: Writing replication status
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /hbase/replication/rs/sv4r23s48,10304,1341112194623/4/sv4r23s48%2C10304%2C1341112194623.1341112195369
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1246)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.setData(RecoverableZooKeeper.java:372)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:655)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:697)
        at org.apache.hadoop.hbase.replication.ReplicationZookeeper.writeReplicationStatus(ReplicationZookeeper.java:470)
        at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:154)
        at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:607)
        at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:368)
{noformat}

It seems to me that just refreshing {{otherRegionServers}} after getting the list of {{currentReplicators}}
would be enough to fix this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message