hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-16095) Add priority to TableDescriptor and priority region open thread pool
Date Thu, 23 Jun 2016 20:29:16 GMT
Enis Soztutar created HBASE-16095:

             Summary: Add priority to TableDescriptor and priority region open thread pool
                 Key: HBASE-16095
                 URL: https://issues.apache.org/jira/browse/HBASE-16095
             Project: HBase
          Issue Type: Bug
            Reporter: Enis Soztutar
            Assignee: Enis Soztutar
             Fix For: 2.0.0, 1.4.0

This is in the similar area with HBASE-15816, and also required with the current secondary
indexing for Phoenix. 

The problem with P secondary indexes is that data table regions depend on index regions to
be able to make progress. Possible distributed deadlocks can be prevented via custom RpcScheduler
+ RpcController configuration via HBASE-11048 and PHOENIX-938. However, region opening also
has the same deadlock situation, because data region open has to replay the WAL edits to the
index regions. There is only 1 thread pool to open regions with 3 workers by default. So if
the cluster is recovering / restarting from scratch, the deadlock happens because some index
regions cannot be opened due to them being in the same queue waiting for data regions to open
(which waits for  RPC'ing to index regions which is not open). This is reproduced in almost
all Phoenix secondary index clusters (mutable table w/o transactions) that we see. 

The proposal is to have a "high priority" region opening thread pool, and have the HTD carry
the relative priority of a table. This maybe useful for other "framework" level tables from
Phoenix, Tephra, Trafodian, etc if they want some specific tables to become online faster.

As a follow up patch, we can also take a look at how this priority information can be used
by the rpc scheduler on the server side or rpc controller on the client side, so that we do
not have to set priorities manually per-operation. 

This message was sent by Atlassian JIRA

View raw message