hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-12841) ClientBackoffPolicies should support immediate rejection of submitted ops
Date Mon, 12 Jan 2015 23:59:34 GMT
Andrew Purtell created HBASE-12841:

             Summary: ClientBackoffPolicies should support immediate rejection of submitted
                 Key: HBASE-12841
                 URL: https://issues.apache.org/jira/browse/HBASE-12841
             Project: HBase
          Issue Type: Improvement
            Reporter: Andrew Purtell

The ClientBackoffPolicy interface currently has a single method:
public interface ClientBackoffPolicy {
  public long getBackoffTime(ServerName serverName, byte[] region, ServerStatistics stats);

A backoff policy can only specify the amount of delay to inject before submitting the request(s)
to a given server. 

How that works in the current implementation is we will submit runnables to AsyncProcess that
sleep for the specified delay period before proceeding. This consumes task slots that could
otherwise be performing useful work. AsyncProcess limits the number of outstanding tasks per
region to "hbase.client.max.perregion.tasks" (default 1) and per server "hbase.client.max.perserver.tasks"
(default 2). Tasks will be accepted and queued up to "hbase.client.max.total.tasks" (default
100), after which we start globally blocking submissions by waiting on a monitor.

Sophisticated applications could benefit from an alternate strategy that immediately rejects
new work. Rather then returning a backoff interval, the policy could return a value from 0.0
to 1.0, or as percentage from 0 to 100, expressing the likelihood of task rejection. Immediately
rejected tasks won't consume task slots nor "stall" by sleeping. Overall the client will be
less likely to hit the global limit. Applications using APIs like Table#batch or Table#batchCallback
will get control back faster, can determine what operations were failed by pushback, and deal
intelligently with request ordering and resubmission/retry concerns. 

This message was sent by Atlassian JIRA

View raw message