hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guanghao Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15529) Override needBalance in StochasticLoadBalancer
Date Fri, 20 May 2016 01:48:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292537#comment-15292537

Guanghao Zhang commented on HBASE-15529:

We use it in our production cluster one week. A use case of our cluster is:
2016-05-20,00:49:17,765 INFO org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer:
start StochasticLoadBalancer.balaner, initCost=22.90110461931783, functionCost=RegionCountSkewCostFunction
: (25.0, 0.0); MoveCostFunction : (5.0, 0.0); LocalityCostFunction : (30.0, 0.6408324325039945);
TableSkewCostFunction : (30.0, 0.018651123682791523); ReadRequestCostFunction : (5.0, 0.4076729529674278);
WriteRequestCostFunction : (5.0, 0.21564663377542329); MemstoreSizeCostFunction : (0.0, 0.13922403222035973);
StoreFileCostFunction : (0.0, 0.12878739127954114); 
RegionCountSkewCost is 0.0 which means all rs have the same number of regions. The original
needsBalance method will return false because all rs have the same number regions. So the
StochasticLoadBalancer will not run even the LocalityCost is 0.64.

The hbase version of our cluster is 0.98, it didn't have the feature of region replica.

> Override needBalance in StochasticLoadBalancer
> ----------------------------------------------
>                 Key: HBASE-15529
>                 URL: https://issues.apache.org/jira/browse/HBASE-15529
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Guanghao Zhang
>            Assignee: Guanghao Zhang
>            Priority: Minor
>         Attachments: HBASE-15529-v1.patch, HBASE-15529.patch
> StochasticLoadBalancer includes cost functions to compute the cost of region rount, r/w
qps, table load, region locality, memstore size, and storefile size. Every cost function returns
a number between 0 and 1 inclusive and the computed costs are scaled by their respective multipliers.
The bigger multiplier means that the respective cost function have the bigger weight. But
needBalance decide whether to balance only by region count and doesn't consider r/w qps, locality
even you config these cost function with bigger multiplier. StochasticLoadBalancer should
override needBalance and decide whether to balance by it's configs of cost functions.
> Add one new config hbase.master.balancer.stochastic.minCostNeedBalance, cluster need
balance when (total cost / sum multiplier) > minCostNeedBalance.

This message was sent by Atlassian JIRA

View raw message