hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chia-Ping Tsai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19924) hbase rpc throttling does not work for multi() with request count rater.
Date Fri, 16 Mar 2018 01:50:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401374#comment-16401374
] 

Chia-Ping Tsai commented on HBASE-19924:
----------------------------------------

*Q1*
 Given that we need to check both of size and count, it should be ok to get rid of following
methods
{code:java}
  /**
   * Checks if it is possible to execute the specified operation.
   *
   * @param estimateWriteSize the write size that will be checked against the available quota
   * @param estimateReadSize the read size that will be checked against the available quota
   * @throws ThrottlingException thrown if not enough avialable resources to perform operation.
   */
  void checkQuota(long estimateWriteSize, long estimateReadSize)
    throws ThrottlingException;

  /**
   * Removes the specified write and read amount from the quota.
   * At this point the write and read amount will be an estimate,
   * that will be later adjusted with a consumeWrite()/consumeRead() call.
   *
   * @param writeSize the write size that will be removed from the current quota
   * @param readSize the read size that will be removed from the current quota
   */
  void grabQuota(long writeSize, long readSize);
{code}

*Q2*
 avialable _> available

*Q3 (unrelated to this jira)*

Why we consume the write/read req by heap size? Should we make both methods to accept both
size and count.
{code:TimeBasedLimiter.java}
@Override
public void consumeWrite(final long size) {
  reqSizeLimiter.consume(size);
  writeSizeLimiter.consume(size);
}

@Override
public void consumeRead(final long size) {
  reqSizeLimiter.consume(size);
  readSizeLimiter.consume(size);
}
{code}

*Q4 (unrelated to this jira)*
Is it necessary to add the condition check (xxxSize > 0)? Is it impossible to have a zero
size op? For example, the Result generated by a check-exist Get may have zero heap size since
the heap size of Result is the sum of heap size of cells.
{code}
    if (estimateWriteSize > 0) {
      if (!writeReqsLimiter.canExecute(writeReqs)) {
        ThrottlingException.throwNumWriteRequestsExceeded(writeReqsLimiter.waitInterval());
      }
      if (!writeSizeLimiter.canExecute(estimateWriteSize)) {
        ThrottlingException.throwWriteSizeExceeded(writeSizeLimiter.waitInterval(estimateWriteSize));
      }
    }

    if (estimateReadSize > 0) {
{code}

> hbase rpc throttling does not work for multi() with request count rater.
> ------------------------------------------------------------------------
>
>                 Key: HBASE-19924
>                 URL: https://issues.apache.org/jira/browse/HBASE-19924
>             Project: HBase
>          Issue Type: Bug
>          Components: rpc
>    Affects Versions: 0.16.0, 1.2.6
>            Reporter: huaxiang sun
>            Assignee: huaxiang sun
>            Priority: Major
>         Attachments: HBASE-19924-master-v001.patch
>
>
> Basically, rpc throttling does not work for request count based rater for multi. for
the following code, when it calls limiter's checkQuota(), numWrites/numReads is lost.
> {code:java}
> @Override
> public void checkQuota(int numWrites, int numReads, int numScans) throws ThrottlingException
{
>   writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
>   readConsumed = estimateConsume(OperationType.GET, numReads, 100);
>   readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
>   writeAvailable = Long.MAX_VALUE;
>   readAvailable = Long.MAX_VALUE;
>   for (final QuotaLimiter limiter : limiters) {
>     if (limiter.isBypass()) continue;
>     limiter.checkQuota(writeConsumed, readConsumed);
>     readAvailable = Math.min(readAvailable, limiter.getReadAvailable());
>     writeAvailable = Math.min(writeAvailable, limiter.getWriteAvailable());
>   }
>   for (final QuotaLimiter limiter : limiters) {
>     limiter.grabQuota(writeConsumed, readConsumed);
>   }
> }{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message