Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B4827200C49 for ; Fri, 17 Mar 2017 08:54:47 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id B32C3160B80; Fri, 17 Mar 2017 07:54:47 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 0689E160B6D for ; Fri, 17 Mar 2017 08:54:46 +0100 (CET) Received: (qmail 70083 invoked by uid 500); 17 Mar 2017 07:54:46 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 70068 invoked by uid 99); 17 Mar 2017 07:54:46 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Mar 2017 07:54:46 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 8E397C0974 for ; Fri, 17 Mar 2017 07:54:45 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.651 X-Spam-Level: X-Spam-Status: No, score=0.651 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_NEUTRAL=0.652] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id qZ4zbZKnhER4 for ; Fri, 17 Mar 2017 07:54:44 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 142D05FCCA for ; Fri, 17 Mar 2017 07:54:44 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id C4C29E09F4 for ; Fri, 17 Mar 2017 07:54:42 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id B6866254BD for ; Fri, 17 Mar 2017 07:54:41 +0000 (UTC) Date: Fri, 17 Mar 2017 07:54:41 +0000 (UTC) From: "Hudson (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-17706) TableSkewCostFunction improperly computes max skew MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 17 Mar 2017 07:54:47 -0000 [ https://issues.apache.org/jira/browse/HBASE-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929552#comment-15929552 ] Hudson commented on HBASE-17706: -------------------------------- FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2687 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2687/]) HBASE-17706 TableSkewCostFunction improperly computes max skew - revert (tedyu: rev a69c23abfeed9246f308bf470b72a9a8afa46f5d) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java > TableSkewCostFunction improperly computes max skew > -------------------------------------------------- > > Key: HBASE-17706 > URL: https://issues.apache.org/jira/browse/HBASE-17706 > Project: HBase > Issue Type: Bug > Components: Balancer > Affects Versions: 1.2.0 > Environment: CentOS Derivative with a derivative of the 3.18.43 kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches. > Reporter: Kahlil Oppenheimer > Assignee: Kahlil Oppenheimer > Priority: Minor > Labels: patch > Fix For: 2.0 > > Attachments: HBASE-17706-01.patch, HBASE-17706-02.patch, HBASE-17706-03.patch, HBASE-17706-04.patch, HBASE-17706-05.patch, HBASE-17706-06.patch, HBASE-17706-07.patch, HBASE-17706.patch > > > We noticed while running unit tests that the TableSkewCostFunction computed cost did not change as the balancer ran and simulated moves across the cluster. After investigating, we found that this happened in particular when the cluster started out with at least one table very strongly skewed. > We noticed that the TableSkewCostFunction depends on a field of the BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field is not properly maintained as regionMoves are simulated for the cluster. The field only ever increases as the maximum number of regions per table increases, but it does not decrease as the maximum number per table goes down. > This patch corrects that behavior so that the field is accurately maintained, and thus the TableSkewCostFunction produces a more correct value as the balancer runs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)