hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13376) Improvements to Stochastic load balancer
Date Tue, 19 May 2015 02:34:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549682#comment-14549682
] 

Ted Yu commented on HBASE-13376:
--------------------------------

The test failure can be reproduced:
{code}
testRebalanceOnRegionServerNumberChange[1](org.apache.hadoop.hbase.TestRegionRebalancing)
 Time elapsed: 8.44 sec  <<< ERROR!
java.lang.NullPointerException: null
	at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster$2.compare(BaseLoadBalancer.java:772)
	at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster$2.compare(BaseLoadBalancer.java:769)
	at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324)
	at java.util.TimSort.sort(TimSort.java:189)
	at java.util.TimSort.sort(TimSort.java:173)
	at java.util.Arrays.sort(Arrays.java:659)
	at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.sortServersByLocality(BaseLoadBalancer.java:762)
	at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.getLowestLocalityRegionServer(BaseLoadBalancer.java:788)
	at org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$LocalityBasedCandidateGenerator.pickLowestLocalityServer(StochasticLoadBalancer.java:625)
	at org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$LocalityBasedCandidateGenerator.generate(StochasticLoadBalancer.java:597)
	at org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:266)
	at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1241)
	at org.apache.hadoop.hbase.TestRegionRebalancing.testRebalanceOnRegionServerNumberChange(TestRegionRebalancing.java:119)
{code}
The NPE seems to come from this line in BaseLoadBalancer$Cluster$2.compare :
{code}
        float locality1 = getLocality(integer);
{code}

> Improvements to Stochastic load balancer
> ----------------------------------------
>
>                 Key: HBASE-13376
>                 URL: https://issues.apache.org/jira/browse/HBASE-13376
>             Project: HBase
>          Issue Type: Improvement
>          Components: Balancer
>    Affects Versions: 1.0.0, 0.98.12
>            Reporter: Vandana Ayyalasomayajula
>            Assignee: Vandana Ayyalasomayajula
>            Priority: Minor
>         Attachments: HBASE-13376_0.98.txt, HBASE-13376_0.txt, HBASE-13376_98.patch
>
>
> There are two things this jira tries to address:
> 1. The locality picker in the stochastic balancer does not pick regions with least locality
as candidates for swap/move. So when any user configures locality cost in the configs, the
balancer does not always seems to move regions with bad locality. 
> 2. When a cluster has equal number of loaded regions, it always picks the first one.
It should pick a random region on one of the equally loaded servers. This improves a chance
of finding a good candidate, when load picker is invoked several times. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message