hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jeff saremi <jeffsar...@hotmail.com>
Subject Re: What is the cause for RegionTooBusyException?
Date Fri, 26 May 2017 17:52:55 GMT
@James, thank you very much. That was extremely helpful

________________________________
From: James Moore <jcmoore@hubspot.com>
Sent: Friday, May 26, 2017 10:24:42 AM
To: user@hbase.apache.org
Subject: Re: What is the cause for RegionTooBusyException?

One mechanism for revealing the error in question is to print one of the
individual exceptions which are included in the batch calls response.  We
use this in a few places to allow inspection of individual Exceptions you
can see an example of how to do this over here
https://github.com/apache/hbase/blob/master/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java#L1222


On Fri, May 26, 2017 at 12:47 PM, jeff saremi <jeffsaremi@hotmail.com>
wrote:

> Hi Stack
>
> no there are no details in the exception. I mentioned that in another
> thread. When you perform a Batch operation, I believe no details will be
> communicated. I am not sure about individual Put's though. That makes it
> hard to go through logs cause we don't know out of hundreds of RS's which
> logs to look at
>
> I have an issue with this exception being thrown period. I think the
> resource management needs a lot of work. I will soon post another note
> about my impression of this whole thing.
>
> Jeff
>
> ________________________________
> From: saint.ack@gmail.com <saint.ack@gmail.com> on behalf of Stack <
> stack@duboce.net>
> Sent: Friday, May 26, 2017 12:05:36 AM
> To: Hbase-User
> Subject: Re: What is the cause for RegionTooBusyException?
>
> On Mon, May 22, 2017 at 9:31 AM, jeff saremi <jeffsaremi@hotmail.com>
> wrote:
>
> > while I'm still trying to find anything useful in the logs, my question
> is
> > why isn't HBase self managing this?
> >
>
> It should do better here, yes (I thought TooBusy retried but I am not
> finding it at the mo.). Exception is thrown for such as the reasons James
> lists -- in essence out of resources --  including the case where we fail
> to obtain lock inside the configured timeouts (row lock on write or region
> lock doing bulk load). As James notes, you should see the too busy dumped
> into the regionserver log at time of issue. Having this, you can figure
> what resource is crimped. Is there no more detail on client side on the
> root of the TooBusy exceptions?
>
>
> Thanks,
> S
>
>
>
> >
> > ________________________________
> > From: jeff saremi <jeffsaremi@hotmail.com>
> > Sent: Friday, May 19, 2017 8:18:59 PM
> > To: user@hbase.apache.org
> > Subject: Re: What is the cause for RegionTooBusyException?
> >
> > Thanks Ted. I will look deeper as you suggested
> >
> > ________________________________
> > From: Ted Yu <yuzhihong@gmail.com>
> > Sent: Friday, May 19, 2017 4:18:12 PM
> > To: user@hbase.apache.org
> > Subject: Re: What is the cause for RegionTooBusyException?
> >
> > Have you checked region server log ?
> > Please take a look at the following method in HRegion:
> >
> >   private void checkResources() throws RegionTooBusyException {
> >
> > ...
> >
> >     if (this.memstoreDataSize.get() > this.blockingMemStoreSize) {
> >
> >       blockedRequestsCount.increment();
> >
> >       requestFlush();
> >
> >       throw new RegionTooBusyException("Above memstore limit, " +
> >
> > Which hbase release are you using ?
> >
> > Cheers
> >
> > On Fri, May 19, 2017 at 3:59 PM, jeff saremi <jeffsaremi@hotmail.com>
> > wrote:
> >
> > > We're getting errors like this. Where should we be looking into to
> solve
> > > this?
> > >
> > >
> > > Failed 69261 actions: RegionTooBusyException: 12695 times,
> > > RemoteWithExtrasException: 56566 times
> > >
> > > thanks
> > >
> > > Jeff
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message