hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wayne <wav...@gmail.com>
Subject Re: memstore flush blocking write pause
Date Mon, 23 May 2011 15:59:08 GMT
We have 4 CFs, but only 1 is ever used for a given region. What about upping
the size per memstore file to 1G? We have 5x limit of 256m which results in
lots of messages like "memstore size 1.3g is >= than blocking 1.2g size".
Maybe given the bigger region size we need a bigger memstore size?

Here is a region server log snippet for this occurring 2x in less than a 2
minute period.

http://pastebin.com/CxAQSXTt


On Mon, May 23, 2011 at 11:33 AM, Stack <stack@duboce.net> wrote:

> On Mon, May 23, 2011 at 6:40 AM, Wayne <wav100@gmail.com> wrote:
> > In order to reduce the total number of regions we have up'd the max
> region
> > size to 5g. This has kept us below 100 regions per node but the side
> affect
> > is pauses occurring every 1-2 min under heavy writes to a single region.
> We
> > see the "too many store files delaying flush up to 90sec" warning every
> > couple of minutes. We have upped the size of the memstore flush size
> (256m)
> > as well as upped the blockingstorefiles (15) but these pauses
> > are occurring more than writes are occurring. In the end our write
> > through-put has degraded considerably.
> >
>
> How many column families?  Pastebin a regionserver log.  You could up
> the number of store files before we put up the blocking writes gate
> but then you might have runaway files to compact.
>
> St.Ack
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message