hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Scan performance on a big table as combination of multiple logic tables
Date Tue, 21 Feb 2012 20:08:58 GMT
On Sun, Feb 19, 2012 at 1:45 PM, Mikael Sitruk <mikael.sitruk@gmail.com> wrote:
> During compaction the region is not out of service.
> According to documentation the max region size for V2 format is 20G
> And now the question: Assuming that 20G is the limit and the number of
> regions in a single RS should stay low < 500 it means that there is no mean
> having RS with more than 10TB of storage to use by HBase (otherwise
> locality will not be achieve for some servers, i also assume that
> compression is used and therefore it compensate the need for additional
> space for replication)?
> If the max number of region per RS is smaller then the storage size is even
> smaller. Is it correct?

In the documentation 20GB is given as an example of a larger size that
can be supported, but nothing blocks you from going way higher than
that. I've done some import tests and had 100GB regions. It just takes
a while to compact the bigger files.

Also you can go over 500 regions, in fact one of our clusters has
14,398 regions right now. It's just a pain to reassign everything when
HBase boots but this is an offline cluster.

J-D

Mime
View raw message