hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13965) Stochastic Load Balancer JMX Metrics
Date Thu, 09 Jul 2015 23:41:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621457#comment-14621457
] 

Lei Chen commented on HBASE-13965:
----------------------------------

I have found a problem related to HBASE-5231(per-table load balancing).
It seems that the balancing is done by an iteration of tables. 
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1219-L1228

This can be configured to be in per-table mode or ensemble mode. 
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java#L956-L962

In ensemble mode, all the tables are copied into an "ensemble" table for balancing. The configuration
is set by
 {{hbase.master.loadbalancer.bytable}}

My question is how to name the metrics if the balancing is in ensemble mode.

For example, suppose we have two tables, "Table1" and "Table2", and N cost functions.
In the per-table mode, each table will have an overall cost and one for each cost function.
    {{Table1_Overall}}
    {{Table1_costFunction}} x N
    {{Table2_Overall}}
    {{Table2_costFunction}} x N
In the ensemble mode, there will be only one overall and one set of function costs.
    {{ensemble_Overall}}
    {{ensemble_costFunction}} x N

Can we use a special name for the combined table, e.g. "ensemble"? The problem is that the
user may have already created a table named "ensemble", which may cause confusion. 

Any idea on this problem?


> Stochastic Load Balancer JMX Metrics
> ------------------------------------
>
>                 Key: HBASE-13965
>                 URL: https://issues.apache.org/jira/browse/HBASE-13965
>             Project: HBase
>          Issue Type: Improvement
>          Components: Balancer, metrics
>            Reporter: Lei Chen
>            Assignee: Lei Chen
>         Attachments: HBASE-13965-v3.patch, HBASE-13965-v4.patch, HBASE-13965-v5.patch,
HBASE-13965-v6.patch, HBASE-13965_v2.patch, HBase-13965-v1.patch, stochasticloadbalancerclasses_v2.png
>
>
> Today’s default HBase load balancer (the Stochastic load balancer) is cost function
based. The cost function weights are tunable but no visibility into those cost function results
is directly provided.
> A driving example is a cluster we have been tuning which has skewed rack size (one rack
has half the nodes of the other few racks). We are tuning the cluster for uniform response
time from all region servers with the ability to tolerate a rack failure. Balancing LocalityCost,
RegionReplicaRack Cost and RegionCountSkew Cost is difficult without a way to attribute each
cost function’s contribution to overall cost. 
> What this jira proposes is to provide visibility via JMX into each cost function of the
stochastic load balancer, as well as the overall cost of the balancing plan.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message