hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Corgan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3373) Allow regions of specific table to be load-balanced
Date Fri, 28 Jan 2011 20:20:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988214#action_12988214
] 

Matt Corgan commented on HBASE-3373:
------------------------------------

Have you guys considered using a consistent hashing method to choose which server a region
belongs to?  You would create ~50 buckets for each server by hashing serverName_port_bucketNum,
and then hash the start key of each region into the buckets.

There are a few benefits:

* when you add a server it takes an equal load from all existing servers
* if you remove a server it distributes its regions equally to the remaining servers
* adding a server does not cause all regions to shuffle like round robin assignment would
* assignment is nearly random, but repeatable, so no hot spots
* when a region splits the front half will stay on the same server, but the back half will
usually be sent to another server

And a few drawbacks:

* each server wouldn't end up with exactly the same number of regions, but they would be close
* if a hot spot does end up developing, you can't do anything about it, at least not unless
it supported a list of manual overrides



> Allow regions of specific table to be load-balanced
> ---------------------------------------------------
>
>                 Key: HBASE-3373
>                 URL: https://issues.apache.org/jira/browse/HBASE-3373
>             Project: HBase
>          Issue Type: Improvement
>          Components: master
>    Affects Versions: 0.20.6
>            Reporter: Ted Yu
>             Fix For: 0.92.0
>
>
> From our experience, cluster can be well balanced and yet, one table's regions may be
badly concentrated on few region servers.
> For example, one table has 839 regions (380 regions at time of table creation) out of
which 202 are on one server.
> It would be desirable for load balancer to distribute regions for specified tables evenly
across the cluster. Each of such tables has number of regions many times the cluster size.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message