Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <2040270885.2727916.1474515183637@mail.yahoo.com>
References: <198413734.2444994.1474478314310.ref@mail.yahoo.com>
 <198413734.2444994.1474478314310@mail.yahoo.com> <CAB8hLa=ZrVtw+Brr1MJgpp1qNY3BZRmi0KbZGec+d5=-MVGGNA@mail.gmail.com>
 <2040270885.2727916.1474515183637@mail.yahoo.com>
From: Guanghao Zhang <zghaobac@gmail.com>
Date: Thu, 22 Sep 2016 11:58:46 +0800
Message-ID: <CAB8hLak1BG2CzgO0KHt8zow7Epbsts=XZ1SQz75TKBBKrLjPbw@mail.gmail.com>
Subject: Re: Hbase throttling issues
To: user@hbase.apache.org, Sumit Nigam <sumit_only@yahoo.com>
Content-Type: multipart/alternative; boundary=001a1143edfcbc779e053d10ade7
archived-at: Thu, 22 Sep 2016 03:58:53 -0000

--001a1143edfcbc779e053d10ade7
Content-Type: text/plain; charset=UTF-8

Did you try to use REQUEST_NUMBER as throttle type?

2016-09-22 11:33 GMT+08:00 Sumit Nigam <sumit_only@yahoo.com.invalid>:

> Hi Guanghao,
>
> This throttling error shows up as soon as I start hbase. So, ideally there
> shouldn't be too many prior operations in play here. Plus, the error shows
> up even when my table has hardly any data (possibly in kbs) and I have set
> throttling limit to be ~10GB.
>
> Thanks,Sumit
>
>       From: Guanghao Zhang <zghaobac@gmail.com>
>  To: user@hbase.apache.org; Sumit Nigam <sumit_only@yahoo.com>
>  Sent: Thursday, September 22, 2016 7:37 AM
>  Subject: Re: Hbase throttling issues
>
> All scan operations were throttled? Now it use the avg size of all previous
> operations to check quota. Maybe the previous scan operation read too much
> data.
>
> 2016-09-22 1:18 GMT+08:00 Sumit Nigam <sumit_only@yahoo.com.invalid>:
>
> > Hi Hbase experts,
> >
> > Our application is unable to scan or read from hbase tables, when
> > throttling is set. We are getting ThrottlingException every time. The
> error
> > is seen more frequently when the number of hbase pre splits is
> > increased. The size tables for which this error is showing is empty (
> > during some runs it was very low in the order of few kbs). Have tried
> both
> > rate limiters already - average and fixed. Can't understand why read rate
> > limit exceeds when there is hardly any data in hbase. Anyone has faced
> this
> > issue before?
> >
> > Setup Details:
> >
> > Hbase version : 1.1.2
> > Number of region servers :4
> > Number of regions : 116
> > HeapMemory for Region Server : 2GB
> >
> > Quotas set :
> > TABLE => ns1:table1 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT
> > => 10G/sec, SCOPE => MACHINE
> > TABLE => ns2:table2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT
> > => 10G/sec, SCOPE => MACHINE
> >
> > Following is the error we faced. Pasting debug logs of regions server:
> >
> > 2016-09-17 22:35:40,674 DEBUG [B.defaultRpcServer.handler=
> 55,queue=1,port=58526]
> > quotas.RegionServerQuotaManager: Throttling exception for user=root
> > table=ns1:table1 numWrites=0 numReads=0 numScans=1: read size limit
> > exceeded - wait 0.00sec
> >
> > 2016-09-17 22:35:40,676 DEBUG [B.defaultRpcServer.handler=
> 55,queue=1,port=58526]
> > ipc.RpcServer: B.defaultRpcServer.handler=55,queue=1,port=58526: callId:
> > 52 service: ClientService methodName: Scan size: 28 connection:
> > 10.65.141.170:42806
> >
> > org.apache.hadoop.hbase.quotas.ThrottlingException: read size limit
> > exceeded - wait 0.00sec
> >
> > at org.apache.hadoop.hbase.quotas.ThrottlingException.
> > throwThrottlingException(ThrottlingException.java:107)
> >
> > at org.apache.hadoop.hbase.quotas.ThrottlingException.
> > throwReadSizeExceeded(ThrottlingException.java:101)
> >
> > at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.
> > checkQuota(TimeBasedLimiter.java:139)
> >
> > at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(
> > DefaultOperationQuota.java:59)
> >
> > at org.apache.hadoop.hbase.quotas.RegionServerQuotaManager.checkQuota(
> > RegionServerQuotaManager.java:180)
> >
> > at org.apache.hadoop.hbase.quotas.RegionServerQuotaManager.checkQuota(
> > RegionServerQuotaManager.java:125)
> >
> > at org.apache.hadoop.hbase.regionserver.RSRpcServices.
> > scan(RSRpcServices.java:2265)
> >
> > at org.apache.hadoop.hbase.protobuf.generated.
> > ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
> >
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
> >
> > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
> >
> > at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(
> > RpcExecutor.java:130)
> >
> > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> >
> > at java.lang.Thread.run(Thread.java:745)
> >
> >
> > Thanks,
> >
> > Sumit
>
>
>
>

--001a1143edfcbc779e053d10ade7--