hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Corgan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3373) Allow regions of specific table to be load-balanced
Date Fri, 28 Jan 2011 20:20:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988214#action_12988214

Matt Corgan commented on HBASE-3373:

Have you guys considered using a consistent hashing method to choose which server a region
belongs to?  You would create ~50 buckets for each server by hashing serverName_port_bucketNum,
and then hash the start key of each region into the buckets.

There are a few benefits:

* when you add a server it takes an equal load from all existing servers
* if you remove a server it distributes its regions equally to the remaining servers
* adding a server does not cause all regions to shuffle like round robin assignment would
* assignment is nearly random, but repeatable, so no hot spots
* when a region splits the front half will stay on the same server, but the back half will
usually be sent to another server

And a few drawbacks:

* each server wouldn't end up with exactly the same number of regions, but they would be close
* if a hot spot does end up developing, you can't do anything about it, at least not unless
it supported a list of manual overrides

> Allow regions of specific table to be load-balanced
> ---------------------------------------------------
>                 Key: HBASE-3373
>                 URL: https://issues.apache.org/jira/browse/HBASE-3373
>             Project: HBase
>          Issue Type: Improvement
>          Components: master
>    Affects Versions: 0.20.6
>            Reporter: Ted Yu
>             Fix For: 0.92.0
> From our experience, cluster can be well balanced and yet, one table's regions may be
badly concentrated on few region servers.
> For example, one table has 839 regions (380 regions at time of table creation) out of
which 202 are on one server.
> It would be desirable for load balancer to distribute regions for specified tables evenly
across the cluster. Each of such tables has number of regions many times the cluster size.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message