hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jianshi Huang <jianshi.hu...@gmail.com>
Subject Re: Need Help: RegionTooBusyException: Above memstore limit
Date Tue, 03 Mar 2015 08:14:02 GMT
Hi JM,

Thanks for the hints. Here's my settings for writer.

edgeTable.setAutoFlush(false)
edgeTable.setWriteBufferSize(20971520)

The write buffer seems quite large as the region server is hosting 12
related regions I'm writing to. I'll test with smaller write buffer size.

The size of each put is between 10k~100k.

Jianshi

On Mon, Mar 2, 2015 at 11:04 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Jo Jianshi,
>
> Are you doing batch of puts? If so, what's the size of the batch and what's
> the size of the puts? Are you trying to give a batch which at the end will
> be bigger than the memstore size for a single RS? Can you try to reduce the
> size of this batch?
>
> JM
>
> 2015-03-02 9:59 GMT-05:00 Ted Yu <yuzhihong@gmail.com>:
>
> > What's the value for hbase.regionserver.global.memstore.size ?
> >
> > Did RegionTooBusyException happen to many regions or only a few regions ?
> >
> > How much heap did you give region servers ?
> >
> > bq. HBase version is 0.98.0.2.1.2.0-402
> >
> > Yeah, this is a bit old. Please consider upgrading.
> >
> > Cheers
> >
> > On Mon, Mar 2, 2015 at 1:42 AM, Jianshi Huang <jianshi.huang@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I'm constantly facing "RegionTooBusyException: Above memstore limit"
> > errors
> > > in one region server when writing data to HBase.
> > >
> > > I checked the region server log, and I've seen a lot of warnings during
> > the
> > > data writes:
> > >
> > >   WARN wal.fshlog couldn't find oldest seqNum for the region we're
> about
> > to
> > > flush, ...
> > >
> > > Then HBase seem to flush the data and added it as a HStore file.
> > >
> > > I also get a few warnings in client.ShortCircuitCache, says "could not
> > load
> > > ... due to InvalidToken exceptions.
> > >
> > > Anyone can give me hint what went wrong?
> > >
> > > My HBase version is 0.98.0.2.1.2.0-402, I'm using HDP 2.1, but the
> > release
> > > is a little bit old.
> > >
> > > Thanks,
> > >
> > > --
> > > Jianshi Huang
> > >
> > > LinkedIn: jianshi
> > > Twitter: @jshuang
> > > Github & Blog: http://huangjs.github.com/
> > >
> >
>



-- 
Jianshi Huang

LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message