hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-2699) Reimplement load balancing to be a background process and to not use heartbeats
Date Thu, 08 Jul 2010 17:31:56 GMT

     [ https://issues.apache.org/jira/browse/HBASE-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Gray updated HBASE-2699:
---------------------------------

    Attachment: HBASE-2699-LoadBalancer-v2.patch

Adds new {{LoadBalancer}} class to the master package.  Class defines three primary methods.

The new load balancing algorithm drops the notion of slop and fully balances the cluster (this
will be a background process so no need to use slop to prevent churn).  The method is a bit
long but I did my best to document it... at least it's in one method and testable now :) 
Balancing uses the following method which takes a map of all the servers to their regions
and returns a list of regions to be moved with their source and destination servers:
{noformat}
public List<RegionPlan> balanceCluster(Map<HServerInfo,List<HRegionInfo>>
serverInfo)
{noformat}

For cluster startup, there is a bulk assignment method.  Currently this is implemented as
round-robin but should eventually be extended to use block locations (there are some methods
in this class to help but not integrated yet):
{noformat}
public Map<HServerInfo,List<HRegionInfo>> bulkAssignment(List<HRegionInfo>
regions, List<HServerInfo> servers)
{noformat}

The other time we may need to determine assignments of regions is after master failover. 
A new master may find regions who have been closed but no longer has in-memory state (a {{RegionPlan}})
to say where it should be opened.  In this case, the master will use an immediate assignment
method, the goal of which is to just come up with an assignment as fast as possible.  Currently
it randomly picks a server for each input region.
{noformat}
public Map<HRegionInfo,HServerInfo> immediateAssignment(List<HRegionInfo> regions,
List<HServerInfo> servers)
{noformat}

Patch also adds a nice fat TestLoadBalancer unit test with tests for each of the three methods.
 It has input arrays for a bunch of mock cluster states and methods to verify the balancing
invariants to ensure correctness.

If anyone has more good edge cases let's get em added!

Yay, a testable load balancer!

> Reimplement load balancing to be a background process and to not use heartbeats
> -------------------------------------------------------------------------------
>
>                 Key: HBASE-2699
>                 URL: https://issues.apache.org/jira/browse/HBASE-2699
>             Project: HBase
>          Issue Type: Sub-task
>          Components: master, regionserver
>            Reporter: Jonathan Gray
>            Assignee: Jonathan Gray
>            Priority: Critical
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2699-LoadBalancer-v2.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message