hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei Chen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-13965) Stochastic Load Balancer JMX Metrics
Date Fri, 10 Jul 2015 20:28:05 GMT

     [ https://issues.apache.org/jira/browse/HBASE-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lei Chen updated HBASE-13965:
-----------------------------
    Attachment: HBASE-13965-v7.patch

Updates:
1. Overloaded balanceCluster() to pass the table name to balancer
2. Moved some string constants to HConstants.java
3. Stochastic balancer auto adjust JMX metrics size by the number of tables
4. Stochastic handles both ensemble and per-table modes.
5. Updated tests to cover both modes.

TODO:
1. The tests currently only use the miniCluster to save and read JMX metrics, which means
that the tables are not actually stored in hbase. I'm NOT sure if this method is adequate
or we need to save real tables to miniCluster and balance them for real?

Sorry guys, I still cannot upload the patch file to review board. The diff file always gets
"No valid separator after the filename was found in the diff header" error. If I manually
touch up the file by adding (revision xxxx) or (working copy), I got "revision xxxx cannot
be found" error. The command line rbt has the same problem.

> Stochastic Load Balancer JMX Metrics
> ------------------------------------
>
>                 Key: HBASE-13965
>                 URL: https://issues.apache.org/jira/browse/HBASE-13965
>             Project: HBase
>          Issue Type: Improvement
>          Components: Balancer, metrics
>            Reporter: Lei Chen
>            Assignee: Lei Chen
>         Attachments: HBASE-13965-v3.patch, HBASE-13965-v4.patch, HBASE-13965-v5.patch,
HBASE-13965-v6.patch, HBASE-13965-v7.patch, HBASE-13965_v2.patch, HBase-13965-v1.patch, stochasticloadbalancerclasses_v2.png
>
>
> Today’s default HBase load balancer (the Stochastic load balancer) is cost function
based. The cost function weights are tunable but no visibility into those cost function results
is directly provided.
> A driving example is a cluster we have been tuning which has skewed rack size (one rack
has half the nodes of the other few racks). We are tuning the cluster for uniform response
time from all region servers with the ability to tolerate a rack failure. Balancing LocalityCost,
RegionReplicaRack Cost and RegionCountSkew Cost is difficult without a way to attribute each
cost function’s contribution to overall cost. 
> What this jira proposes is to provide visibility via JMX into each cost function of the
stochastic load balancer, as well as the overall cost of the balancing plan.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message