hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "M. C. Srivas" <mcsri...@gmail.com>
Subject Re: Scan performance on a big table as combination of multiple logic tables
Date Wed, 22 Feb 2012 01:44:40 GMT
On Tue, Feb 21, 2012 at 12:08 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> On Sun, Feb 19, 2012 at 1:45 PM, Mikael Sitruk <mikael.sitruk@gmail.com>
> wrote:
> > During compaction the region is not out of service.
> > According to documentation the max region size for V2 format is 20G
> > And now the question: Assuming that 20G is the limit and the number of
> > regions in a single RS should stay low < 500 it means that there is no
> mean
> > having RS with more than 10TB of storage to use by HBase (otherwise
> > locality will not be achieve for some servers, i also assume that
> > compression is used and therefore it compensate the need for additional
> > space for replication)?
> > If the max number of region per RS is smaller then the storage size is
> even
> > smaller. Is it correct?
>
> In the documentation 20GB is given as an example of a larger size that
> can be supported, but nothing blocks you from going way higher than
> that. I've done some import tests and had 100GB regions. It just takes
> a while to compact the bigger files.
>

With no impact on Java GC going nuts?  FB reported (a few months ago) it
was bad to run a region-server
with -Xmx larger than 15G or 16G. Unless its no longer true, wouldn't that
be limiting factor for how
large one should make regions?





>
> Also you can go over 500 regions, in fact one of our clusters has
> 14,398 regions right now. It's just a pain to reassign everything when
> HBase boots but this is an offline cluster.
>
> J-D
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message