hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Blocking of callers during compaction
Date Tue, 18 Aug 2009 14:33:56 GMT
Mathias,

There is indeed something to do. At least, we should not clear the
buffer if the put fails for some reason.

WRT to the blocking, last night Stack and Jon fixed an issue where we
were blocking too much (HBASE-1771). I guess that's what you hit since
reclaimMemStoreMemory will flush regions when it's able to do so. So
updating to RC2 should help your case. Also, if you have enough RAM,
you could change hbase.regionserver.global.memstore.upperLimit and
lowerLimit to something higher since 40% of 1GB is very different than
40% of 4GB. Then you can also set a higher
hbase.hstore.blockingStoreFiles.

J-D

On Tue, Aug 18, 2009 at 10:13 AM, Mathias
Herberts<mathias.herberts@gmail.com> wrote:
> I've encountered a situation lately where lots of regions need
> compaction at the same time. This leads to callers being blocked
> waiting to grab the monitor to enter 'reclaimMemStoreMemory'.
>
> When the number of regions is large, the blocking can last a long time.
>
> In my specific case, the callers are Reducer instances and the timeout
> of the reducer (mapred.task.timeout) actually gets reached and the
> Reducer gets restarted.
>
> I was thinking about a possible enhancement where we could specify in
> HTable that in case of compaction we want to be pushed back
> immediately instead of being blocked.
>
> This could be done by adding a boolean to Put objects that would be
> set in HTable.put(), reflecting the HTable setting (thus avoiding to
> have several values of this flag in a list of Put). The modification
> of this flag would trigger a flushCommits so all Put instances in the
> buffer carry the same value of the flag.
>
> The behavior in case a request is pushed back would be to throw an
> IOException, leaving the write buffer as it was prior to the call.
>
> This would allow in my case the Reducer to wait on its side, modifying
> its status so mapred.task.timeout is not reahed.
>
> What do you think?
>
> Mathias.
>

Mime
View raw message