hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-6721) RegionServer Group based Assignment
Date Tue, 20 Dec 2016 07:19:59 GMT

     [ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-6721:
-------------------------
    Release Note: 
[ADVANCED USERS ONLY] This patch adds a new experimental module hbase-rsgroup. It is an advanced
feature for partitioning regionservers into distinctive groups for strict isolation, and should
only be used by users who are sophisticated enough to understand the full implications and
have a sufficient background in managing HBase clusters. 

RSGroups can be defined and managed with shell commands or corresponding Java APIs. A server
can be added to a group with hostname and port pair, and tables can be moved to this group
so that only regionservers in the same rsgroup can host the regions of the table. RegionServers
and tables can only belong to 1 group at a time. By default, all tables and regionservers
belong to the "default" group. System tables can also be put into a group using the regular
APIs. A custom balancer implementation tracks assignments per rsgroup and makes sure to move
regions to the relevant regionservers in that group. The group information is stored in a
regular HBase table, and a zookeeper-based read-only cache is used at the cluster bootstrap
time. 

To enable, add the following to your hbase-site.xml and restart your Master:

{code}
<property>
  <name>hbase.coprocessor.master.classes</name>
  <value>org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint</value>
</property>
<property>
  <name>hbase.master.loadbalancer.class</name>
  <value>org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer</value>
</property>
{code}

Then use the shell 'rsgroup' commands to create and manipulate regionserver groups: e.g. to
add a group and then add a server to it, do as follows:

{code}
hbase(main):008:0> add_rsgroup 'my_group'
Took 0.5610 seconds
{code}

This adds a group to the 'hbase:rsgroup' system table. Add a server (hostname + port) to the
group using the 'move_rsgroup_servers' command as follows:

{code}
move_rsgroup_servers 'my_group',['k.att.net:51129']
{code}

  was:
[ADVANCED USERS ONLY] This patch adds a new experimental module hbase-rsgroup. It is an advanced
feature for partitioning regionservers into distinctive groups for strict isolation, and should
only be used by users who are sophisticated enough to understand the full implications and
have a sufficient background in managing HBase clusters. 

RSGroups can be defined and managed with shell commands or corresponding Java APIs. A server
can be added to a group with hostname and port pair, and tables can be moved to this group
so that only regionservers in the same rsgroup can host the regions of the table. RegionServers
and tables can only belong to 1 group at a time. By default, all tables and regionservers
belong to the "default" group. System tables can also be put into a group using the regular
APIs. A custom balancer implementation tracks assignments per rsgroup and makes sure to move
regions to the relevant regionservers in that group. The group information is stored in a
regular HBase table, and a zookeeper-based read-only cache is used at the cluster bootstrap
time. 


> RegionServer Group based Assignment
> -----------------------------------
>
>                 Key: HBASE-6721
>                 URL: https://issues.apache.org/jira/browse/HBASE-6721
>             Project: HBase
>          Issue Type: New Feature
>          Components: regionserver
>            Reporter: Francis Liu
>            Assignee: Francis Liu
>              Labels: hbase-6721
>             Fix For: 2.0.0
>
>         Attachments: 6721-master-webUI.patch, HBASE-6721 GroupBasedLoadBalancer Sequence
Diagram.xml, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, HBASE-6721_12.patch, HBASE-6721_13.patch,
HBASE-6721_14.patch, HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, HBASE-6721_9.patch,
HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch,
HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, HBASE-6721_94_7.patch,
HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch,
HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, HBASE-6721_trunk2.patch,
balanceCluster Sequence Diagram.svg, hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch,
hbase-6721-v17.patch, hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, hbase-6721-v21.patch,
hbase-6721-v22.patch, hbase-6721-v23.patch, hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch,
hbase-6721-v27.patch, hbase-6721-v27.patch, hbase-6721-v27.patch.txt, hbase-6721-v28.patch,
hbase-6721-v28.patch, hbase-6721-v29.patch, immediateAssignments Sequence Diagram.svg, randomAssignment
Sequence Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment Sequence
Diagram.svg
>
>
> In multi-tenant deployments of HBase, it is likely that a RegionServer will be serving
out regions from a number of different tables owned by various client applications. Being
able to group a subset of running RegionServers and assign specific tables to it, provides
a client application a level of isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of RegionServer
groups and assigns tables to region servers based on groupings. Load balancing will occur
on a per group basis as well. 
> This is essentially a simplification of the approach taken in HBASE-4120. See attached
document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message