hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13965) Stochastic Load Balancer JMX Metrics
Date Tue, 30 Jun 2015 02:15:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606864#comment-14606864

Lei Chen commented on HBASE-13965:

Sorry for the delay. I propose the following changes which can be seen illustrated in the
attached class diagram.

The {{StochasticLoadBalancer}} extends {{BaseLoadBalancer}} which has a class variable of
type {{MetricsBalancer}}. The {{MetricsBalancer}} contains a private class variable of type
{{MetricsBalancerSource}}. The {{MetricsBalancerSource}} is an interface which defines which
metrics can be reported to JMX. This proves to make extension difficult for load balancer
implementation specific metrics (e.g. the {{StochasticLoadBalancer}}).

Adding metrics to the generic interface is not appropriate being it is used by all load balancers
and should not contain any load balancer specific metrics. I propose to create a class extending
{{MetricsBalancer}} to provide specific load balancer metrics. To use this class, I propose
to add a constructor to {{BaseLoadBalancer}} which allows for the balancer instance metrics
class to be passed in. (Thanks [~enis] for code review and giving the constructor suggestion!)

In the constructor of {{StochasticLoadBalancer}}, an instance of {{MetricsStochasticBalancer}}
is created and passed to a new constructor added to {{BaseLoadBalancer}}, which will use it
to replace the default {{MetricsBalancer}}. The function used to add metrics is declared as
    public void updateStochasticCost(String tableName, String costFunctionName, String costFunctionDesc,
Double value);

In {{MetricsBalancer}}, the {{private final}} class variable {{source}} was previously hardcoded
and instantiated in its constructor; I propose a new function {{initSource}} which can be
overridden to set this variable. As such, in the subclass {{MetricsStochasticBalancer}}, {{initSource}}
will create a {{MetricsStochasticBalancerSource}} instance instead of the default {{MetricsBalancerSource}}.

Finally, to give good insight to the internal status of {{StochasticLoadBalancer}}, we are
considering adding metrics for each cost function, as well as the overall cost. For example,
if the balancing is carried out for table "MyTable1" and the {{StochasticLoadBalancer}} has
3 cost functions "MoveCost", "LocalityCost", and "RegionReplicaHostCost", then 4 metrics will
be added to “HBase -> Master -> Balancer” as following:

I'm building the patch, any suggestion is appreciated.

> Stochastic Load Balancer JMX Metrics
> ------------------------------------
>                 Key: HBASE-13965
>                 URL: https://issues.apache.org/jira/browse/HBASE-13965
>             Project: HBase
>          Issue Type: Improvement
>          Components: Balancer, metrics
>            Reporter: Lei Chen
>            Assignee: Lei Chen
> Today’s default HBase load balancer (the Stochastic load balancer) is cost function
based. The cost function weights are tunable but no visibility into those cost function results
is directly provided.
> A driving example is a cluster we have been tuning which has skewed rack size (one rack
has half the nodes of the other few racks). We are tuning the cluster for uniform response
time from all region servers with the ability to tolerate a rack failure. Balancing LocalityCost,
RegionReplicaRack Cost and RegionCountSkew Cost is difficult without a way to attribute each
cost function’s contribution to overall cost. 
> What this jira proposes is to provide visibility via JMX into each cost function of the
stochastic load balancer, as well as the overall cost of the balancing plan.

This message was sent by Atlassian JIRA

View raw message