hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jianshi Huang <jianshi.hu...@gmail.com>
Subject Re: Need Help: RegionTooBusyException: Above memstore limit
Date Tue, 03 Mar 2015 16:10:33 GMT
Yes, looks like reducing the batch buffer size works (still validating).

But why setAutoFlush(false) is harmful here? I just want maximum write
speed.

Jianshi

On Tue, Mar 3, 2015 at 10:54 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Let HBase manage the flushes for you. Remove edgeTable.setAutoFlush(false)
> and maybe reduce your batch size.
>
> I don't think that increasing the memstore is the good way to go. Sound
> more like a plaster on the issue than a good fix (for me).
>
> JM
>
> 2015-03-03 9:43 GMT-05:00 Ted Yu <yuzhihong@gmail.com>:
>
> > Default value for hbase.regionserver.global.memstore.size is 0.4
> >
> > Meaning Maximum size of all memstores in the region server before new
> > updates
> > are blocked and flushes are forced is 7352m which is lower than 774m.
> >
> > You can increase the value for hbase.regionserver.global.memstore.size
> >
> > Please also see if you can distribute the writes to the underlying region
> > so that the region's use of memstore comes down.
> >
> > Cheersx
> >
> > On Tue, Mar 3, 2015 at 12:07 AM, Jianshi Huang <jianshi.huang@gmail.com>
> > wrote:
> >
> > > Hi Ted,
> > >
> > > Only one region server is problematic.
> > >
> > > hbase.regionserver.global.memstore.size is not set, the problematic
> > region
> > > is using 774m for memstore.
> > >
> > > Max heap is 18380m for all region servers.
> > >
> > > Jianshi
> > >
> > >
> > > On Mon, Mar 2, 2015 at 10:59 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > >
> > > > What's the value for hbase.regionserver.global.memstore.size ?
> > > >
> > > > Did RegionTooBusyException happen to many regions or only a few
> > regions ?
> > > >
> > > > How much heap did you give region servers ?
> > > >
> > > > bq. HBase version is 0.98.0.2.1.2.0-402
> > > >
> > > > Yeah, this is a bit old. Please consider upgrading.
> > > >
> > > > Cheers
> > > >
> > > > On Mon, Mar 2, 2015 at 1:42 AM, Jianshi Huang <
> jianshi.huang@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I'm constantly facing "RegionTooBusyException: Above memstore
> limit"
> > > > errors
> > > > > in one region server when writing data to HBase.
> > > > >
> > > > > I checked the region server log, and I've seen a lot of warnings
> > during
> > > > the
> > > > > data writes:
> > > > >
> > > > >   WARN wal.fshlog couldn't find oldest seqNum for the region we're
> > > about
> > > > to
> > > > > flush, ...
> > > > >
> > > > > Then HBase seem to flush the data and added it as a HStore file.
> > > > >
> > > > > I also get a few warnings in client.ShortCircuitCache, says "could
> > not
> > > > load
> > > > > ... due to InvalidToken exceptions.
> > > > >
> > > > > Anyone can give me hint what went wrong?
> > > > >
> > > > > My HBase version is 0.98.0.2.1.2.0-402, I'm using HDP 2.1, but the
> > > > release
> > > > > is a little bit old.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > --
> > > > > Jianshi Huang
> > > > >
> > > > > LinkedIn: jianshi
> > > > > Twitter: @jshuang
> > > > > Github & Blog: http://huangjs.github.com/
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Jianshi Huang
> > >
> > > LinkedIn: jianshi
> > > Twitter: @jshuang
> > > Github & Blog: http://huangjs.github.com/
> > >
> >
>



-- 
Jianshi Huang

LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message