hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2699) Reimplement load balancing to be a background process and to not use heartbeats
Date Thu, 08 Jul 2010 18:43:52 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886403#action_12886403

Jonathan Gray commented on HBASE-2699:

The load balancer will be triggered according to some configurable period, like every 5 minutes
(if you don't mind more churn) or once every hour (to keep churn down).

The first thing it does is check that the cluster is in fact "out of balance", and without
slop this is strict.  We used to not have slop but it was introduced to prevent load balancing
from getting triggered after just one or two splits threw things just off balance.  As a periodic
background process, we won't be sensitive to things like this and instead will do the balancing
on a fixed period.

There could be some improvements later that force a load balance if the cluster reaches a
certain level of "out of balance" but that's an optimization, not considering it now.

This balancer should eventually move towards something cost-based that takes a much more complete
picture of "load" into account.  I'm not sure a notion of "slop" makes sense in this context...
the goal will always be to balance as best as possible (with a side-goal of not moving regions
around if it doesn't improve the balance... meaning, if a region could be on X or Y server
and achieve the same balance, it should bias towards staying where it is, minimal churn).
 The algo will certainly need an overhaul when we implement this.

In any case, slop adds unnecessary complexity and the original purpose it was added is no
longer relevant IMO.  With this new balancer, it would mean overloaded servers would always
hover at the top of the slop range, underloaded servers would hover at the bottom of the slop
range.  Why not actually balance?  Simpler algorithm.

I also expect region movement to be quite fast with the double-flush and zk-based transitions.
 Hopefully have some real timings next week.

> Reimplement load balancing to be a background process and to not use heartbeats
> -------------------------------------------------------------------------------
>                 Key: HBASE-2699
>                 URL: https://issues.apache.org/jira/browse/HBASE-2699
>             Project: HBase
>          Issue Type: Sub-task
>          Components: master, regionserver
>            Reporter: Jonathan Gray
>            Assignee: Jonathan Gray
>            Priority: Critical
>             Fix For: 0.21.0
>         Attachments: HBASE-2699-LoadBalancer-v2.patch

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message