hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francis Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
Date Fri, 28 Aug 2015 18:47:49 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720395#comment-14720395

Francis Liu commented on HBASE-6721:

All of this has already been tried at FB and it was a mistake. This ends up looking functionally
very similar to 0.89-fb's favored nodes. ( Only assign regions to specific set of machines
that's configured by the admin ). It's so bad that almost every time we try and solve an issue
on a cluster with favored nodes, the first thing we do is turn off the balancer so that we
don't have to worry about which nodes are configured to have with regions. That's literally
step one of debugging. Turn off this feature.
We'll have a party when FB no longer has this operational nightmare. I won't sign anyone up
for the same. I won't sign myself up for the same.
IMHO that's not an objective comparison. Favored Nodes and Region Server groups are very different.
Their use cases are very different and their implementations are also very different.  

As for how useful it is for us (and potenitally for others), if we actually removed region
server groups I'm pretty sure our HBase team and SEs would revolt :-). If we didn't have this
feature we would be managing around 80 hbase clusters right now instead of the 6 multi-tenant
cluster we are currently running. Step one of debugging is not turning of the balancer that
would make things worse. In fact one of the useful features of region server groups is quickly
isolating tables to a new group if they are misbehaving or their workload has changed. This
can be done in a few minutes if not seconds. 

Assignment manager is already too complex adding more complexity is awful
If you look at the patch, the change in AM is an extra 20 lines of code. 6 lines are just
bugfixes that should be done to AM anyway and the other 14 lines which is fairly straightforward
we can even live without if that's what it takes.

region movement is already too stateful. Adding more is awful
Adding more states?

Configuration of HBase is already way too complex. Multiplying that with multiple groups is
Not sure what the concern here is? That there's an option to configure a different balancer?

> RegionServer Group based Assignment
> -----------------------------------
>                 Key: HBASE-6721
>                 URL: https://issues.apache.org/jira/browse/HBASE-6721
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Francis Liu
>            Assignee: Francis Liu
>              Labels: hbase-6721
>         Attachments: 6721-master-webUI.patch, HBASE-6721 GroupBasedLoadBalancer Sequence
Diagram.xml, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf,
HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, HBASE-6721_12.patch, HBASE-6721_8.patch,
HBASE-6721_9.patch, HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721_94_2.patch,
HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, HBASE-6721_94_4.patch, HBASE-6721_94_5.patch,
HBASE-6721_94_6.patch, HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch,
HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk.patch,
HBASE-6721_trunk1.patch, HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, immediateAssignments
Sequence Diagram.svg, randomAssignment Sequence Diagram.svg, retainAssignment Sequence Diagram.svg,
roundRobinAssignment Sequence Diagram.svg
> In multi-tenant deployments of HBase, it is likely that a RegionServer will be serving
out regions from a number of different tables owned by various client applications. Being
able to group a subset of running RegionServers and assign specific tables to it, provides
a client application a level of isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of RegionServer
groups and assigns tables to region servers based on groupings. Load balancing will occur
on a per group basis as well. 
> This is essentially a simplification of the approach taken in HBASE-4120. See attached

This message was sent by Atlassian JIRA

View raw message