hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sheng Chen <chensheng2...@gmail.com>
Subject Re: memstore flush blocking write pause
Date Tue, 14 Jun 2011 06:21:45 GMT
Thank you JD.

My hbase works as a log archive storage, there is a continuing stream of
inserting.
So it is normally write heavy. Is there any advice for this scenario?
Thanks.

Sean



2011/6/14 Jean-Daniel Cryans <jdcryans@apache.org>

> Unless your normal workload is very heavy on writes (which is Wayne's
> case), you're better off using bulk loading:
> http://hbase.apache.org/bulk-loads.html
>
> J-D
>
> On Mon, Jun 13, 2011 at 12:26 AM, Sheng Chen <chensheng2010@gmail.com>
> wrote:
> > I've met with the same problem.
> > Update operations are blocked by memstore flushing, and memstore flushing
> is
> > blocked by a compaction ("too many store files, delay flushing for 90s").
> >
> > Have you got any solutions?
> >
> > 2011/5/23 Wayne <wav100@gmail.com>
> >
> >> We have 4 CFs, but only 1 is ever used for a given region. What about
> >> upping
> >> the size per memstore file to 1G? We have 5x limit of 256m which results
> in
> >> lots of messages like "memstore size 1.3g is >= than blocking 1.2g
> size".
> >> Maybe given the bigger region size we need a bigger memstore size?
> >>
> >> Here is a region server log snippet for this occurring 2x in less than a
> 2
> >> minute period.
> >>
> >> http://pastebin.com/CxAQSXTt
> >>
> >>
> >> On Mon, May 23, 2011 at 11:33 AM, Stack <stack@duboce.net> wrote:
> >>
> >> > On Mon, May 23, 2011 at 6:40 AM, Wayne <wav100@gmail.com> wrote:
> >> > > In order to reduce the total number of regions we have up'd the max
> >> > region
> >> > > size to 5g. This has kept us below 100 regions per node but the side
> >> > affect
> >> > > is pauses occurring every 1-2 min under heavy writes to a single
> >> region.
> >> > We
> >> > > see the "too many store files delaying flush up to 90sec" warning
> every
> >> > > couple of minutes. We have upped the size of the memstore flush size
> >> > (256m)
> >> > > as well as upped the blockingstorefiles (15) but these pauses
> >> > > are occurring more than writes are occurring. In the end our write
> >> > > through-put has degraded considerably.
> >> > >
> >> >
> >> > How many column families?  Pastebin a regionserver log.  You could up
> >> > the number of store files before we put up the blocking writes gate
> >> > but then you might have runaway files to compact.
> >> >
> >> > St.Ack
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message