hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guorui Wu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
Date Fri, 07 Aug 2015 23:55:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662670#comment-14662670

Guorui Wu commented on HBASE-6721:

As I've been studying this code, in order to aid with the understanding of HBASE-6721, I’ve
created some UML sequence diagrams of functions within GroupBasedLoadBalancer.java. This UML
diagram only focuses on GroupBasedLoadBalancer, since GroupBasedLoadBalancer, along with GroupInfoManager,
GroupInfoManagerImpl and GroupInfo as well as some way to configure groups represent the core
functionality needed for this implementation. This represents the 1,500 core lines implementing
the region server group functionality; this documentation does not cover the extensive configuration
management via CLI stored in Zookeeper and an HBase table.

The goal of {{GroupBasedLoadBalancer}} is to separate servers and regions into groups, and
to balance within groups using another load balancer (referred to as internal load balancer).
The default load balancer used is {{StochasticLoadBalancer}}. Each group is stored as a {{GroupInfo}}
object, and {{GroupInfoManagerImpl}} holds a collection of {{GroupInfo}} objects with APIs
allowing for the manipulation of this collection. These groups can be created and configured
via the CLI, and are stored within an HBase table as well as Zookeeper. As {{GroupBasedLoadBalancer}}
implements the {{LoadBalancer}} interface, it has the following functions from the interface:

As GroupBasedLoadBalancer implements the LoadBalancer interface, it has the following functions
from the interface: 
•	balanceCluster
•	roundRobinAssignment
•	retainAssignment
•	immediateAssignment
•	randomAssignment
•	initialize
•	isStopped
•	setGroupInfoManager

Additionally, GroupBasedLoadBalancer has several helper functions:
•	offlineRetainAssignment
•	onlineRetainAssignment
•	generateGroupMaps
•	filterOfflineServers
•	filterServers
•	getMisplacedRegions
•	correctAssignments
•	isOnline
•	getGroupInfoManager

Through reading the code for the load balancer portion of HBASE-6721, I have a few questions:
1.	Within the interface GroupInfoManager.java, I noticed that the function getGroupOfServer
returns a GroupInfo object, but the function getGroupOfTable returns a String object. Was
there a performance consideration or some other reason for returning a string? (It seems the
API would be more consistent to return a GroupInfo object.)
2.	For the function onlineRetainAssignment why are regions assigned to bogus so it ends up
in RIT if a server is not available? (We would like to keep as few regions in RIT as possible
in order to maximize our availability.)
3.	On the topic of onlineRetainAssignment, what is the objective for separating online and
offline servers? I noticed that another balancer such as the StochasticLoadBalancer does not
make such a distinction.

Lastly, the UML diagram I created can be edited by downloading the attached XML file and editing
with http://draw.io.

> RegionServer Group based Assignment
> -----------------------------------
>                 Key: HBASE-6721
>                 URL: https://issues.apache.org/jira/browse/HBASE-6721
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Francis Liu
>            Assignee: Francis Liu
>         Attachments: 6721-master-webUI.patch, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721_10.patch, HBASE-6721_11.patch,
HBASE-6721_8.patch, HBASE-6721_9.patch, HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch,
HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, HBASE-6721_94_4.patch,
HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, HBASE-6721_94_7.patch, HBASE-6721_trunk.patch,
HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, HBASE-6721_trunk2.patch
> In multi-tenant deployments of HBase, it is likely that a RegionServer will be serving
out regions from a number of different tables owned by various client applications. Being
able to group a subset of running RegionServers and assign specific tables to it, provides
a client application a level of isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of RegionServer
groups and assigns tables to region servers based on groupings. Load balancing will occur
on a per group basis as well. 
> This is essentially a simplification of the approach taken in HBASE-4120. See attached

This message was sent by Atlassian JIRA

View raw message