hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerry He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10926) Use global procedure to flush table memstore cache
Date Fri, 02 May 2014 01:36:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987246#comment-13987246
] 

Jerry He commented on HBASE-10926:
----------------------------------

bq. you don't reject concurrent flush on the same table, is that on purpose?

Concurrent flush on the same table is rejected.

{code}
    // We rely on the existing Distributed Procedure framework to prevent any concurrent
    // procedure with the same name.
    Procedure proc = coordinator.startProcedure(monitor, desc.getInstance(),
      new byte[0], Lists.newArrayList(regionServers));
    monitor.rethrowException();
    if (proc == null) {
      String msg = "Failed to submit distributed procedure " + desc.getSignature() + " for
'"
          + desc.getInstance() + "'. " + "Another flush procedure is running?";
      LOG.error(msg);
      throw new IOException(msg);
    }
{code}

In the master log:
{code}
2014-05-01 11:15:26,630 INFO org.apache.hadoop.hbase.master.HMaster: Client=biadmin//9.30.194.18
procedure request for: flush-table-proc
2014-05-01 11:15:26,644 WARN org.apache.hadoop.hbase.procedure.ProcedureCoordinator: Procedure
table2 currently running.  Rejecting new request
2014-05-01 11:15:26,644 ERROR org.apache.hadoop.hbase.procedure.ProcedureCoordinator: Failed
to submit procedure 'table2'
2014-05-01 11:15:26,644 ERROR org.apache.hadoop.hbase.procedure.flush.MasterFlushTableProcedureManager:
Failed to submit distributed procedure flush-table-proc for 'table2'. Another flush procedure
is running?
2014-05-01 11:15:26,852 INFO org.apache.hadoop.hbase.master.HMaster: Client=biadmin//9.30.194.18
procedure request for: flush-table-proc
2014-05-01 11:15:26,869 WARN org.apache.hadoop.hbase.procedure.ProcedureCoordinator: Procedure
table2 currently running.  Rejecting new request
2014-05-01 11:15:26,869 ERROR org.apache.hadoop.hbase.procedure.ProcedureCoordinator: Failed
to submit procedure 'table2'
2014-05-01 11:15:26,869 ERROR org.apache.hadoop.hbase.procedure.flush.MasterFlushTableProcedureManager:
Failed to submit distributed procedure flush-table-proc for 'table2'. Another flush procedure
is running?
{code}

Interestingly, in HBaseAdmin executeCallable, which execProcedure() uses , is RPC with retries.
So in the log you can see the client just keeps trying until the previous flush procedure
completes.
Then it will be able to run on the same table.

{code}
2014-05-01 11:15:38,788 INFO org.apache.hadoop.hbase.master.HMaster: Client=biadmin//9.30.194.18
procedure request for: flush-table-proc
2014-05-01 11:15:38,804 DEBUG org.apache.hadoop.hbase.procedure.ProcedureCoordinator: Procedure
table2 was in running list but was completed.  Accepting new attempt.
2014-05-01 11:15:38,804 INFO org.apache.hadoop.hbase.procedure.Procedure: Starting procedure
'table2'
2014-05-01 11:15:38,804 DEBUG org.apache.hadoop.hbase.errorhandling.TimeoutExceptionInjector:
Scheduling process timer to run in: 60000 ms
2014-05-01 11:15:38,804 DEBUG org.apache.hadoop.hbase.procedure.Procedure: Procedure 'table2'
starting 'acquire'
...
{code}
So effectively concurrent requests from the client will be executed sequentially.

> Use global procedure to flush table memstore cache
> --------------------------------------------------
>
>                 Key: HBASE-10926
>                 URL: https://issues.apache.org/jira/browse/HBASE-10926
>             Project: HBase
>          Issue Type: Improvement
>          Components: Admin
>    Affects Versions: 0.96.2, 0.98.1
>            Reporter: Jerry He
>            Assignee: Jerry He
>             Fix For: 0.99.0
>
>         Attachments: HBASE-10926-trunk-v1.patch, HBASE-10926-trunk-v2.patch
>
>
> Currently, user can trigger table flush through hbase shell or HBaseAdmin API.  To flush
the table cache, each region server hosting the regions is contacted and flushed sequentially,
which is less efficient.
> In HBase snapshot global procedure is used to coordinate and flush the regions in a distributed
way.
> Let's provide a distributed table flush for general use.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message