hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Custom preCompact RegionObserver crashes entire cluster on OOME: Heap Space
Date Tue, 12 Feb 2013 05:53:01 GMT
It would be nice if you can try your improved code using 0.94.4 where
HBASE-7205 <https://issues.apache.org/jira/browse/HBASE-7205> is included.


On Mon, Feb 11, 2013 at 9:46 PM, Mesika, Asaf <asaf.mesika@gmail.com> wrote:

> Hi,
> I wrote a RegionObserver which does preCompact.
> I activated in pre-production, and then entire cluster dropped dead: One
> RegionServer after another crashed on OutOfMemoryException: Heap Space.
> My preCompact method generates a KeyValue per each set of Column
> Qualifiers it sees.
> When I remove the coprocessor and restart the cluster, cluster remains
> stable.
> I have 8 RS, each has 4 GB Heap. There about 9 regions (from a specific
> table I'm working on) per Region Server.
> Running HBase 0.94.3
> The crash occur when the major compaction fires up, apparently cluster
> wide.
> My question is this: Where do the KV generated during the compaction
> process queue up before being written to the disk? Is this buffer
> configurable?
> When I wrote the Region Observer my assumption was the the compaction
> process works in Streaming fashion, thus even if I decide to generate a KV
> per KV I see, it still shouldn't be a problem memory wise.
> Of course I'm trying to improve my code so it will generate much less new
> KV (by simply altering the existing KVs received from the InternalScanner).
> Thank you,
> Asaf

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message