hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wang Qiang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
Date Thu, 25 Apr 2013 03:52:17 GMT

     [ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wang Qiang updated HBASE-8432:
------------------------------

    Status: Patch Available  (was: Open)

Index: DefaultLoadBalancer.java

===================================================================

--- DefaultLoadBalancer.java	(revision 87270)

+++ DefaultLoadBalancer.java	(working copy)

@@ -318,7 +318,14 @@

     fetchFromTail = false;
 
     Map<ServerName, Integer> underloadedServers = new HashMap<ServerName, Integer>();
-    int maxToTake = numRegions - (int)average;
+	int maxToTake = 0;
+	for (Map.Entry<ServerAndLoad, List<HRegionInfo>> server : serversByLoad
+			.entrySet()) {
+		int regionCount = server.getKey().getLoad();
+		if (regionCount > 1 && regionCount > average) {
+			maxToTake += regionCount - (int) Math.ceil(average);
+		}
+	}
     for (Map.Entry<ServerAndLoad, List<HRegionInfo>> server:
         serversByLoad.entrySet()) {
       if (maxToTake == 0) break; // no more to take

                
> a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-8432
>                 URL: https://issues.apache.org/jira/browse/HBASE-8432
>             Project: HBase
>          Issue Type: Bug
>          Components: Balancer
>    Affects Versions: 0.94.5
>         Environment: Linux 2.6.32-el5.x86_64
>            Reporter: Wang Qiang
>            Priority: Critical
>
> it happened that a table with unbalanced regions, as follows in my cluster(the cluster
has 20 regionservers, the table has 12 regions):
> http://hadoopdev19.cm6:60030/	1
> http://hadoopdev8.cm6:60030/	2
> http://hadoopdev17.cm6:60030/	1
> http://hadoopdev12.cm6:60030/	1
> http://hadoopdev5.cm6:60030/	1
> http://hadoopdev9.cm6:60030/	1
> http://hadoopdev22.cm6:60030/	1
> http://hadoopdev11.cm6:60030/	1
> http://hadoopdev21.cm6:60030/	1
> http://hadoopdev16.cm6:60030/	1
> http://hadoopdev10.cm6:60030/	1
> with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances,
the table are still unbalanced:
> http://hadoopdev3.cm6:60030/	1
> http://hadoopdev20.cm6:60030/	1
> http://hadoopdev4.cm6:60030/	2
> http://hadoopdev18.cm6:60030/	1
> http://hadoopdev12.cm6:60030/	1
> http://hadoopdev14.cm6:60030/	1
> http://hadoopdev15.cm6:60030/	1
> http://hadoopdev6.cm6:60030/	1
> http://hadoopdev13.cm6:60030/	1
> http://hadoopdev11.cm6:60030/	1
> http://hadoopdev10.cm6:60030/	1
> http://hadoopdev19.cm6:60030/	1
> http://hadoopdev17.cm6:60030/	1
> http://hadoopdev8.cm6:60030/	1
> http://hadoopdev5.cm6:60030/	1
> http://hadoopdev12.cm6:60030/	1
> http://hadoopdev22.cm6:60030/	1
> http://hadoopdev11.cm6:60030/	1
> http://hadoopdev21.cm6:60030/	1
> http://hadoopdev7.cm6:60030/	2
> http://hadoopdev10.cm6:60030/	1
> http://hadoopdev16.cm6:60030/	1
> http://hadoopdev3.cm6:60030/	1
> http://hadoopdev20.cm6:60030/	1
> http://hadoopdev4.cm6:60030/	1
> http://hadoopdev18.cm6:60030/	2
> http://hadoopdev12.cm6:60030/	1
> http://hadoopdev14.cm6:60030/	1
> http://hadoopdev15.cm6:60030/	1
> http://hadoopdev6.cm6:60030/	1
> http://hadoopdev13.cm6:60030/	1
> http://hadoopdev11.cm6:60030/	1
> http://hadoopdev10.cm6:60030/	1
> http://hadoopdev19.cm6:60030/	1
> http://hadoopdev8.cm6:60030/	1
> http://hadoopdev17.cm6:60030/	1
> http://hadoopdev12.cm6:60030/	1
> http://hadoopdev5.cm6:60030/	1
> http://hadoopdev22.cm6:60030/	1
> http://hadoopdev11.cm6:60030/	1
> http://hadoopdev7.cm6:60030/	1
> http://hadoopdev21.cm6:60030/	2
> http://hadoopdev16.cm6:60030/	1
> http://hadoopdev10.cm6:60030/	1
> http://hadoopdev3.cm6:60030/	1
> http://hadoopdev20.cm6:60030/	1
> http://hadoopdev18.cm6:60030/	1
> http://hadoopdev4.cm6:60030/	1
> http://hadoopdev12.cm6:60030/	1
> http://hadoopdev15.cm6:60030/	1
> http://hadoopdev14.cm6:60030/	2
> http://hadoopdev6.cm6:60030/	1
> http://hadoopdev13.cm6:60030/	1
> http://hadoopdev11.cm6:60030/	1
> http://hadoopdev10.cm6:60030/	1
> from the above logs, we can also find that some regions needn't move, but they moved.
follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found
that 'maxToTake' is error calculated. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message