hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Spiegelberg <nspiegelb...@fb.com>
Subject Re: region size/count per regionserver
Date Tue, 01 Nov 2011 22:57:03 GMT
Simple answer
20 regions/server & <2000 regions/cluster is a good rule of thumb if you
can't profile your workload yet.  You really want to ensure that

1) You need to limits the regions/cluster so the master can have a
reasonable startup time & can handle all the region state transitions via
ZK.  Most bigger companies are running 2,000 in production and achieve
reasonable startup times (< 2 minutes for region assignment on cold
start).  If you want to test the scalability of that algorithm beyond what
other companies need, admin beware.
2) The more regions/server you have, the faster that recovery can happen
after RS death because you can currently parallelize recovery on a
region-granularity.  Too many regions/server and #1 starts to be a problem.

Complicated answer
More information is optimize this formula.  Additional considerations:

1) Are you IO-bound or CPU-bound
2) What is your grid topology like
3) What is your network hardware like
4) How many disks (not just size)
5) What is the data locality between RegionServer & DataNode

In the Facebook case, we have 5 racks with 20 nodes each.  Servers in the
rack are connected by 1G Eth to a switch with a 10G uplink.  We are
network bound.  Our saturation point is mostly commonly on the top-of-rack
switch.  With 20 regions/server, we can roughly parallelize our
distributed log splitting within a single rack on RS death (although 2
regions do split off-rack).  This minimizes top-of-rack traffic and
optimized our recovery time.  Even if you are CPU-bound, log splitting
(hence recovery time) is an IO-bound operation.  A lot of our work on
region assignment is about maximizing data locality, even on RS death, so
we avoid top-of-rack saturation.

On 11/1/11 10:54 AM, "Sujee Maniyam" <sujee@sujee.net> wrote:

>HI all,
>My HBase cluster is 10 nodes, each node has 12core ,   48G RAM, 24TB disk,
>My region size is 1GB.
>Any guidelines on how many regions can a RS  handle comfortably?
>I vaguely remember reading some where to have no more than 1000 regions /
>server; that comes to 1TB / server.  Seems pretty low for the current
>hardware config.
>Any rules of thumb?  experiences?

View raw message