hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-3406) Region stuck in transition after RS failed while opening
Date Wed, 05 Jan 2011 17:08:51 GMT

     [ https://issues.apache.org/jira/browse/HBASE-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-3406:
-------------------------

    Fix Version/s:     (was: 0.90.0)
                   0.90.1

Moving to 0.90.1

I cannot explain how in-memory state has OPENING for the node but the znode content is M_ZK_REGION_OFFLINE
without more context.

> Region stuck in transition after RS failed while opening
> --------------------------------------------------------
>
>                 Key: HBASE-3406
>                 URL: https://issues.apache.org/jira/browse/HBASE-3406
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.90.0
>            Reporter: Todd Lipcon
>            Priority: Critical
>             Fix For: 0.90.1
>
>
> I had a RS fail due to GC pause while it was in the midst of opening a region, apparently.
This got the region stuck in the following repeating sequence in the master log:
> 2011-01-03 17:24:33,884 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OPENING for too long, reassigning region=usertable,user991629466,1293747979500.c6a54b4d07a44e113b3a4d2ab22daa70.
> 2011-01-03 17:24:33,885 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: master:60000-0x12ce26f6c0600e3
Retrieved 113 byte(s) of data from znode /hbase/unassigned/c6a54b4d07a44e113b3a4d2ab22daa70;
data=region=usertable,user991629466,1293747979500.c6a54b4d07a44e113b3a4d2ab22daa70., server=haus03.sf.cloudera.com:60000,
state=M_ZK_REGION_OFFLINE
> 2011-01-03 17:24:43,886 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  usertable,user991629466,1293747979500.c6a54b4d07a44e113b3a4d2ab22daa70.
state=OPENING, ts=1293840977790
> 2011-01-03 17:24:43,886 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OPENING for too long, reassigning region=usertable,user991629466,1293747979500.c6a54b4d07a44e113b3a4d2ab22daa70.
> 2011-01-03 17:24:43,887 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: master:60000-0x12ce26f6c0600e3
Retrieved 113 byte(s) of data from znode /hbase/unassigned/c6a54b4d07a44e113b3a4d2ab22daa70;
data=region=usertable,user991629466,1293747979500.c6a54b4d07a44e113b3a4d2ab22daa70., server=haus03.sf.cloudera.com:60000,
state=M_ZK_REGION_OFFLINE
> etc... repeating every 10 seconds. Eventually I ran hbck -fix which forced it to OFFLINE
in ZK and it reassigned just fine.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message