hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8119) Optimize StochasticLoadBalancer
Date Tue, 19 Mar 2013 00:03:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13605827#comment-13605827
] 

Enis Soztutar commented on HBASE-8119:
--------------------------------------

bq. Per table load balancing runs the balancer once per table. 
The issue turned out to be not in the per-table load balancing, which was already defaulted
to false. The issue is that for 500 regions, Load balancer takes 15 min, which makes it unusable.
In it's current form, StochasticLoadBalancer can only work with clusters having ~20 nodes,
and low hundreds of regions. 
bq. There's a lot of hashmap manipulation that should be optimized out if we wanted to worry
about perf.
If the balancer takes more than 15 min, there is a bug in HMaster.balance() that it breaks
prematurely from assigning the region plans from the balancer.
One more thing is that we do not do bulk assign to the regions generated by the load balancer
plan. 
                
> Optimize StochasticLoadBalancer
> -------------------------------
>
>                 Key: HBASE-8119
>                 URL: https://issues.apache.org/jira/browse/HBASE-8119
>             Project: HBase
>          Issue Type: Bug
>          Components: Region Assignment
>    Affects Versions: 0.95.0
>            Reporter: Enis Soztutar
>             Fix For: 0.95.0
>
>
> On a 5 node trunk cluster, I ran into a weird problem with StochasticLoadBalancer:
> server1 	Thu Mar 14 03:42:50 UTC 2013 	0.0 	33
> server2 	Thu Mar 14 03:47:53 UTC 2013 	0.0 	34
> server3 	Thu Mar 14 03:46:53 UTC 2013 	465.0 	42
> server4 	Thu Mar 14 03:47:53 UTC 2013 	11455.0 	282
> server5 	Thu Mar 14 03:47:53 UTC 2013 	0.0 	34
> Total:5 		11920 	425
> Notice that server4 has 282 regions, while the others have much less. Plus for one table
with 260 regions has been super imbalanced:
> {code}
> Regions by Region Server
> Region Server	Region Count
> http://server3:60030/ 	10
> http://server4:60030/ 	250
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message