hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: memstore flush blocking write pause
Date Tue, 14 Jun 2011 21:26:27 GMT
Wayne mentions a few of them in his original message in this thread,
also please have a look at this chapter of the online book:
http://hbase.apache.org/book/performance.html

J-D

On Mon, Jun 13, 2011 at 11:21 PM, Sheng Chen <chensheng2010@gmail.com> wrote:
> Thank you JD.
>
> My hbase works as a log archive storage, there is a continuing stream of
> inserting.
> So it is normally write heavy. Is there any advice for this scenario?
> Thanks.
>
> Sean
>
>
>
> 2011/6/14 Jean-Daniel Cryans <jdcryans@apache.org>
>
>> Unless your normal workload is very heavy on writes (which is Wayne's
>> case), you're better off using bulk loading:
>> http://hbase.apache.org/bulk-loads.html
>>
>> J-D
>>
>> On Mon, Jun 13, 2011 at 12:26 AM, Sheng Chen <chensheng2010@gmail.com>
>> wrote:
>> > I've met with the same problem.
>> > Update operations are blocked by memstore flushing, and memstore flushing
>> is
>> > blocked by a compaction ("too many store files, delay flushing for 90s").
>> >
>> > Have you got any solutions?
>> >
>> > 2011/5/23 Wayne <wav100@gmail.com>
>> >
>> >> We have 4 CFs, but only 1 is ever used for a given region. What about
>> >> upping
>> >> the size per memstore file to 1G? We have 5x limit of 256m which results
>> in
>> >> lots of messages like "memstore size 1.3g is >= than blocking 1.2g
>> size".
>> >> Maybe given the bigger region size we need a bigger memstore size?
>> >>
>> >> Here is a region server log snippet for this occurring 2x in less than a
>> 2
>> >> minute period.
>> >>
>> >> http://pastebin.com/CxAQSXTt
>> >>
>> >>
>> >> On Mon, May 23, 2011 at 11:33 AM, Stack <stack@duboce.net> wrote:
>> >>
>> >> > On Mon, May 23, 2011 at 6:40 AM, Wayne <wav100@gmail.com> wrote:
>> >> > > In order to reduce the total number of regions we have up'd the
max
>> >> > region
>> >> > > size to 5g. This has kept us below 100 regions per node but the
side
>> >> > affect
>> >> > > is pauses occurring every 1-2 min under heavy writes to a single
>> >> region.
>> >> > We
>> >> > > see the "too many store files delaying flush up to 90sec" warning
>> every
>> >> > > couple of minutes. We have upped the size of the memstore flush
size
>> >> > (256m)
>> >> > > as well as upped the blockingstorefiles (15) but these pauses
>> >> > > are occurring more than writes are occurring. In the end our write
>> >> > > through-put has degraded considerably.
>> >> > >
>> >> >
>> >> > How many column families?  Pastebin a regionserver log.  You could
up
>> >> > the number of store files before we put up the blocking writes gate
>> >> > but then you might have runaway files to compact.
>> >> >
>> >> > St.Ack
>> >> >
>> >>
>> >
>>
>

Mime
View raw message