Return-Path: <issues-return-269873-archive-asf-public=cust-asf.ponee.io@hbase.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id A7471200BC3
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Fri, 18 Nov 2016 18:10:00 +0100 (CET)
Received: by cust-asf.ponee.io (Postfix)
	id A5730160B03; Fri, 18 Nov 2016 17:10:00 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id 01C8B160B16
	for <archive-asf-public@cust-asf.ponee.io>; Fri, 18 Nov 2016 18:09:59 +0100 (CET)
Received: (qmail 60112 invoked by uid 500); 18 Nov 2016 17:09:59 -0000
Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@hbase.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@hbase.apache.org>
List-Post: <mailto:issues@hbase.apache.org>
List-Id: <issues.hbase.apache.org>
Delivered-To: mailing list issues@hbase.apache.org
Received: (qmail 59978 invoked by uid 99); 18 Nov 2016 17:09:59 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 18 Nov 2016 17:09:59 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id A80292C4C7C
	for <issues@hbase.apache.org>; Fri, 18 Nov 2016 17:09:58 +0000 (UTC)
Date: Fri, 18 Nov 2016 17:09:58 +0000 (UTC)
From: "Yu Li (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13021079.1479286079000.314314.1479488998685@Atlassian.JIRA>
In-Reply-To: <JIRA.13021079.1479286079000@Atlassian.JIRA>
References: <JIRA.13021079.1479286079000@Atlassian.JIRA> <JIRA.13021079.1479286079952@arcas>
Subject: [jira] [Commented] (HBASE-17110) Add an "Overall Strategy"
 option(balanced both on table level and server level) to SimpleLoadBalancer
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Fri, 18 Nov 2016 17:10:00 -0000


    [ https://issues.apache.org/jira/browse/HBASE-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677159#comment-15677159 ] 

Yu Li commented on HBASE-17110:
-------------------------------

Thanks [~enis] for chiming in.

After a second thought, agree to improve the byTable mode directly, and we should locate the root cause of unexpected big batch of region moving and get that resolved.

We've found some points to improve for region assignment during RS crash and create/delete/truncate table, please update the status here later [~xharlie], thanks.

> Add an "Overall Strategy" option(balanced both on table level and server level) to SimpleLoadBalancer
> -----------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-17110
>                 URL: https://issues.apache.org/jira/browse/HBASE-17110
>             Project: HBase
>          Issue Type: New Feature
>          Components: Balancer
>    Affects Versions: 2.0.0, 1.2.4
>            Reporter: Charlie Qiangeng Xu
>            Assignee: Charlie Qiangeng Xu
>         Attachments: HBASE-17110-V2.patch, HBASE-17110-V3.patch, HBASE-17110.patch
>
>
> This jira is about an enhancement of simpleLoadBalancer. Here we introduce a new strategy: "bytableOverall" which could be controlled by adding:
> {noformat}
> <property>
>   <name>hbase.master.loadbalance.bytableOverall</name>
>   <value>true</value>
> </property>
> {noformat}
> We have been using the strategy on our largest cluster for several months. it's proven to be very helpful and stable, especially, the result is quite visible to the users.
> Here is the reason why it's helpful:
> When operating large scale clusters(our case), some companies still prefer to use {{SimpleLoadBalancer}} due to its simplicity, quick balance plan generation, etc. Current SimpleLoadBalancer has two modes: 
> 1. byTable, which only guarantees that the regions of one table could be uniformly distributed. 
> 2. byCluster, which ignores the distribution within tables and balance the regions all together.
> If the pressures on different tables are different, the first byTable option is the preferable one in most case. Yet, this choice sacrifice the cluster level balance and would cause some servers to have significantly higher load, e.g. 242 regions on server A but 417 regions on server B.(real world stats)
> Consider this case,  a cluster has 3 tables and 4 servers:
> {noformat}
>   server A has 3 regions: table1:1, table2:1, table3:1
>   server B has 3 regions: table1:2, table2:2, table3:2
>   server C has 3 regions: table1:3, table2:3, table3:3
>   server D has 0 regions.
> {noformat}
> From the byTable strategy's perspective, the cluster has already been perfectly balanced on table level. But a perfect status should be like:
> {noformat}
>   server A has 2 regions: table2:1, table3:1
>   server B has 2 regions: table1:2, table3:2
>   server C has 3 regions: table1:3, table2:3, table3:3
>   server D has 2 regions: table1:1, table2:2
> {noformat}
> We can see the server loads change from 3,3,3,0 to 2,2,3,2, while the table1, table2 and table3 still keep balanced.   
> And this is what the new mode "byTableOverall" can achieve.
> Two UTs have been added as well and the last one demonstrates the advantage of the new strategy.
> Also, a onConfigurationChange method has been implemented to hot control the "slop" variable.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)