hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Antonov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13103) [ergonomics] add region size balancing as a feature of master
Date Wed, 08 Apr 2015 06:53:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484842#comment-14484842

Mikhail Antonov commented on HBASE-13103:

Thanks for review guys!

bq. I'd like this to eventually be a globally enabled feature, with opt-out via table configuration.
For it's initial commit, it should probably be opt-in instead. Having a global kill switch
is probably a good idea too.

How about adding a boolean flag it table descriptor (false by default) to bring a table to
normalizers attention? Global switch would be still in place to turn everything off if so
desired (probably just like with balancer, there shoud be admin rpc call to turn balancer

.bq Yes, priorities will become a useful feature. I think what you have here is a nice, committable
first pass though.
Thanks, yeah - that would probably be subsequent patch (as well as running in parallel on
all tables opted-in)

.bq Why no touch regions with fewer than 3 regions?
2 considerations I had in mind - tables with fewer than 3 regions probably too small to be
hot spots in terms of cluster throughout (that may be not true?), and also they would require
more thought-out rules. Let's say you have table with 2 regions, 10 and 70 gb. This current
logic would say - well, 70 is < than 40 * 2, so no normalization for you :) What you may
really want is 2 regions of 40, or 5 regions by 16. Need to have "ideal" region size? 

> [ergonomics] add region size balancing as a feature of master
> -------------------------------------------------------------
>                 Key: HBASE-13103
>                 URL: https://issues.apache.org/jira/browse/HBASE-13103
>             Project: HBase
>          Issue Type: Brainstorming
>          Components: Usability
>            Reporter: Nick Dimiduk
>            Assignee: Mikhail Antonov
>             Fix For: 2.0.0, 1.1.0
>         Attachments: HBASE-13103-v0.patch
> Often enough, folks miss-judge split points or otherwise end up with a suboptimal number
of regions. We should have an automated, reliable way to "reshape" or "balance" a table's
region boundaries. This would be for tables that contain existing data. This might look like:
> {noformat}
> Admin#reshapeTable(TableName, int numSplits);
> {noformat}
> or from the shell:
> {noformat}
> > reshape TABLE, numSplits
> {noformat}
> Better still would be to have a maintenance process, similar to the existing Balancer
that runs AssignmentManager on an interval, to run the above "reshape" operation on an interval.
That way, the cluster will automatically self-correct toward a desirable state.

This message was sent by Atlassian JIRA

View raw message