Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A4437200D00 for ; Thu, 3 Aug 2017 03:44:05 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id A2E6416AA6A; Thu, 3 Aug 2017 01:44:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EAA1016AA6D for ; Thu, 3 Aug 2017 03:44:04 +0200 (CEST) Received: (qmail 55976 invoked by uid 500); 3 Aug 2017 01:44:04 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 55871 invoked by uid 99); 3 Aug 2017 01:44:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Aug 2017 01:44:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 30C14C1A21 for ; Thu, 3 Aug 2017 01:44:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id a6mMjakkeGYF for ; Thu, 3 Aug 2017 01:44:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id A6CE65FB62 for ; Thu, 3 Aug 2017 01:44:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 06042E0DFA for ; Thu, 3 Aug 2017 01:44:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 5147624654 for ; Thu, 3 Aug 2017 01:44:00 +0000 (UTC) Date: Thu, 3 Aug 2017 01:44:00 +0000 (UTC) From: "stack (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HBASE-17706) TableSkewCostFunction improperly computes max skew MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 03 Aug 2017 01:44:05 -0000 [ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17706: -------------------------- Priority: Major (was: Minor) > TableSkewCostFunction improperly computes max skew > -------------------------------------------------- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer > Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. > Reporter: Kahlil Oppenheimer > Assignee: Kahlil Oppenheimer > Labels: patch > Fix For: 2.0.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed cost did not change as the balancer ran and simulated moves across the cluster. After investigating, we found that this happened in particular when the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field is not properly maintained as regionMoves are simulated for the cluster. The field only ever increases as the maximum number of regions per table increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, and thus the TableSkewCostFunction produces a more correct value as the balancer runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)