hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerry He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13935) Orphaned namespace table ZK node should not prevent master to start
Date Fri, 19 Jun 2015 19:12:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14593813#comment-14593813

Jerry He commented on HBASE-13935:

2015-05-30 16:56:41,481 INFO  [ProcedureExecutorThread-0] zookeeper.ZKTableStateManager: Moving
table hbase:namespace state from ENABLED to ENABLING
2015-05-30 16:56:41,486 INFO  [ProcedureExecutorThread-0] master.AssignmentManager: Assigning
1 region(s) to ip-666-77-88-999.ec2.internal,16020,1433004979527
2015-05-30 16:56:41,509 INFO  [ProcedureExecutorThread-0] master.RegionStates: Transition
{9df720124fbb66974e95de65bddaee82 state=OFFLINE, ts=1433005001494, server=null} to {9df720124fbb66974e95de65bddaee82
state=PENDING_OPEN, ts=1433005001509, server=ip-666-77-88-999.ec2.internal,16020,1433004979527}
2015-05-30 16:56:41,872 INFO  [ProcedureExecutorThread-0] zookeeper.ZKTableStateManager: Moving
table hbase:namespace state from ENABLING to ENABLED
This is from your unit test run, so we started with ENABLED?
I think the patch looks ok.
We already did a prior check that the table hbase:namespace does not exist in meta accessor.
So we won't mess up there. There may be table files on the filesystem? Then we will fail later.

> Orphaned namespace table ZK node should not prevent master to start
> -------------------------------------------------------------------
>                 Key: HBASE-13935
>                 URL: https://issues.apache.org/jira/browse/HBASE-13935
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 1.0.0, 0.98.13
>            Reporter: Stephen Yuan Jiang
>            Assignee: Stephen Yuan Jiang
>             Fix For: 0.98.14, 1.0.2
>         Attachments: HBASE-13935.v1-0.98.patch, HBASE-13935.v1-branch-1.0.patch
> Before we have the state-of-art Procedure V2 feature (HBASE 1.0 release or older), we
frequently see the following issue (orphaned ZK node) that prevent master to start (at least
in testing):
> {noformat}
> 2015-06-16 17:54:36,472 FATAL [master:] master.HMaster: Unhandled exception.
Starting shutdown.
> org.apache.hadoop.hbase.TableExistsException: hbase:namespace
> 	at org.apache.hadoop.hbase.master.handler.CreateTableHandler.prepare(CreateTableHandler.java:137)
> 	at org.apache.hadoop.hbase.master.TableNamespaceManager.createNamespaceTable(TableNamespaceManager.java:232)
> 	at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:86)
> 	at org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:1123)
> 	at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:947)
> 	at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:618)
> 	at java.lang.Thread.run(Thread.java:745)
> 2015-06-16 17:54:36,472 INFO  [master:] master.HMaster: Aborting
> {noformat}
> The above call trace is from a 0.98.x test run.  We saw similar issue in 1.0.x run, too.
> The proposed fix is to ignore the zk node and force namespace table creation to be complete
so that master can start successfully.  

This message was sent by Atlassian JIRA

View raw message