hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-14190) Assign hbase:namespace table ahead of user region assignment
Date Thu, 06 Aug 2015 16:33:05 GMT
Ted Yu created HBASE-14190:
------------------------------

             Summary: Assign hbase:namespace table ahead of user region assignment
                 Key: HBASE-14190
                 URL: https://issues.apache.org/jira/browse/HBASE-14190
             Project: HBase
          Issue Type: Bug
            Reporter: Ted Yu


Currently the namespace table region is assigned like user regions.

I spent several hours working with a customer where master couldn't finish initialization.
Even though master was restarted quite a few times, it went down with the following:
{code}
2015-08-05 17:16:57,530 FATAL [hdpmaster1:60000.activeMasterManager] master.HMaster: Master
server abort: loaded coprocessors are: []
2015-08-05 17:16:57,530 FATAL [hdpmaster1:60000.activeMasterManager] master.HMaster: Unhandled
exception. Starting shutdown.
java.io.IOException: Timedout 300000ms waiting for namespace table to be assigned
  at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:104)
  at org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:985)
  at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:779)
  at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
  at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
  at java.lang.Thread.run(Thread.java:744)
{code}
During previous run(s), namespace table was created, hence leaving an entry in hbase:meta.
The following if block in TableNamespaceManager#start() was skipped:
{code}
    if (!MetaTableAccessor.tableExists(masterServices.getConnection(),
      TableName.NAMESPACE_TABLE_NAME)) {
{code}
TableNamespaceManager#start() spins, waiting for namespace region to be assigned.
There was issue in master assigning user regions.
We tried issuing 'assign' command from hbase shell which didn't work because of the following
check in MasterRpcServices#assignRegion():
{code}
      master.checkInitialized();
{code}
This scenario can be avoided if we assign hbase:namespace table after hbase:meta is assigned
but before user table region assignment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message