hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2758) META region stuck in RS2ZK_REGION_OPENED state
Date Wed, 23 Jun 2010 03:21:51 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881528#action_12881528
] 

Jonathan Gray commented on HBASE-2758:
--------------------------------------

Patch looks good.  This also fixes an existing race condition where the master node in ZK
was put up before the master got the listing of regionservers.  Nothing would be blocking
the RS from putting up their ephemeral nodes so it was possible that an HMaster thought it
was a failover but it was a clean startup.  Test added in patch verifies that a cluster will
startup even if there are unassigned znodes in zookeeper.

Running full test suite and then will commit.

> META region stuck in RS2ZK_REGION_OPENED state
> ----------------------------------------------
>
>                 Key: HBASE-2758
>                 URL: https://issues.apache.org/jira/browse/HBASE-2758
>             Project: HBase
>          Issue Type: Bug
>          Components: master, regionserver
>    Affects Versions: 0.21.0
>            Reporter: Todd Lipcon
>            Assignee: Karthik Ranganathan
>            Priority: Blocker
>         Attachments: HBASE-2758-0.21.patch
>
>
> In cluster testing trunk, I ended up with a situation where META was unassigned and no
amount of restarting various pieces would fix it. On master startup, I see:
> 2010-06-20 21:08:05,431 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current assignment
of .META.,,1.1028785192 is not valid;  serverAddress=, startCode=0 unknown.
> 2010-06-20 21:08:05,436 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: While
creating UNASSIGNED region 1028785192 exists, state = RS2ZK_REGION_OPENED
> 2010-06-20 21:08:05,438 WARN org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <monster01.sf.cloudera.com:/hbase,org.apache.hadoop.hbase.master.HMaster>Failed
to create ZNode /hbase/UNASSIGNED/1028785192 in ZooKeeper
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
for /hbase/UNASSIGNED/1028785192
> 2010-06-20 21:08:05,438 DEBUG org.apache.hadoop.hbase.master.RegionManager: Created UNASSIGNED
zNode .META.,,1.1028785192 in state M2ZK_REGION_OFFLINE
> then on the RS:
> 2010-06-20 21:08:05,899 ERROR org.apache.hadoop.hbase.regionserver.RSZookeeperUpdater:
ZNode /hbase/UNASSIGNED/1028785192 is not in CLOSED/OFFLINE state (state = RS2ZK_REGION_OPENED),
will NOT open region.
> 2010-06-20 21:08:05,899 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Error
opening .META.,,1.1028785192
> java.io.IOException: ZNode /hbase/UNASSIGNED/1028785192 is not in CLOSED/OFFLINE state
(state = RS2ZK_REGION_OPENED), will NOT open region.
> and the region never opens

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message