Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D10BD18E25 for ; Wed, 12 Aug 2015 00:03:08 +0000 (UTC) Received: (qmail 55547 invoked by uid 500); 12 Aug 2015 00:02:46 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 55433 invoked by uid 500); 12 Aug 2015 00:02:46 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 55203 invoked by uid 99); 12 Aug 2015 00:02:45 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Aug 2015 00:02:45 +0000 Date: Wed, 12 Aug 2015 00:02:45 +0000 (UTC) From: "Biju Nair (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HBASE-14215) Default cost used for PrimaryRegionCountSkewCostFunction is not sufficient MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Biju Nair created HBASE-14215: --------------------------------- Summary: Default cost used for PrimaryRegionCountSkewCostFunction is not sufficient Key: HBASE-14215 URL: https://issues.apache.org/jira/browse/HBASE-14215 Project: HBase Issue Type: Bug Components: Balancer Reporter: Biju Nair Priority: Minor Current multiplier of 500 used in the stochastic balancer cost function `PrimaryRegionCountSkewCostFunction` to calculate the cost of total primary replication skew doesn't seem to be sufficient to prevent the skews (Refer HBASE-14110). We would want the default cost to be a higher value so that skews in primary region replica has higher cost. The following is the test result by setting the multiplier value to 10000 (same as the region replica rack cost multiplier) on a 3 Rack 9 RS node cluster which seems to get the balancer distribute the primaries uniformly. **Initial Primary replica distribution - using the current multiplier** r1n10 102 r1n11 85 r1n9 88 r2n10 120 r2n11 120 r2n9 124 r3n10 135 r3n11 124 r3n9 129 **After long duration of read & writes - using current multiplier** r1n10 102 r1n11 85 r1n9 88 r2n10 120 r2n11 120 r2n9 124 r3n10 135 r3n11 124 r3n9 129 **After manual balancing** r1n10 102 r1n11 85 r1n9 88 r2n10 120 r2n11 120 r2n9 124 r3n10 135 r3n11 124 r3n9 129 **Increased multiplier for primaryRegionCountSkewCost to 10000** r1n10 114 r1n11 113 r1n9 114 r2n10 114 r2n11 114 r2n9 113 r3n10 115 r3n11 115 r3n9 115 Setting the `PrimaryRegionCountSkewCostFunction` multiplier value to 10000 should help HBase general use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)