Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8055A200CD3 for ; Fri, 28 Jul 2017 12:02:06 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 7ECDE16C90E; Fri, 28 Jul 2017 10:02:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id CBA7A16C908 for ; Fri, 28 Jul 2017 12:02:05 +0200 (CEST) Received: (qmail 64025 invoked by uid 500); 28 Jul 2017 10:02:04 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 64013 invoked by uid 99); 28 Jul 2017 10:02:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Jul 2017 10:02:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id C0FD61A1A46 for ; Fri, 28 Jul 2017 10:02:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id pQeotZy-4zqs for ; Fri, 28 Jul 2017 10:02:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 2498261067 for ; Fri, 28 Jul 2017 10:02:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 7ED7EE0E1B for ; Fri, 28 Jul 2017 10:02:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 983A624DB9 for ; Fri, 28 Jul 2017 10:02:00 +0000 (UTC) Date: Fri, 28 Jul 2017 10:02:00 +0000 (UTC) From: "Hudson (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-15134) Add visibility into Flush and Compaction queues MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 28 Jul 2017 10:02:06 -0000 [ https://issues.apache.org/jira/browse/HBASE-15134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16104744#comment-16104744 ] Hudson commented on HBASE-15134: -------------------------------- FAILURE: Integrated in Jenkins build HBase-1.4 #826 (See [https://builds.apache.org/job/HBase-1.4/826/]) HBASE-15134 Add visibility into Flush and Compaction queues (achouhan: rev 92780371080a341d0b6f98307a0ea176db327c5a) * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSource.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperStub.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapper.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegion.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperImpl.java * (edit) hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionSourceImpl.java > Add visibility into Flush and Compaction queues > ----------------------------------------------- > > Key: HBASE-15134 > URL: https://issues.apache.org/jira/browse/HBASE-15134 > Project: HBase > Issue Type: New Feature > Components: Compaction, metrics, regionserver > Reporter: Elliott Clark > Assignee: Abhishek Singh Chouhan > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2 > > Attachments: HBASE-15134.branch-1.001.patch, HBASE-15134.branch-1.001.patch, HBASE-15134.master.001.patch, HBASE-15134.master.002.patch, HBASE-15134.master.003.patch, HBASE-15134.patch, HBASE-15134.patch > > > On busy spurts we can see regionservers start to see large queues for compaction. It's really hard to tell if the server is queueing a lot of compactions for the same region, lots of compactions for lots of regions, or just falling behind. > For flushes much the same. There can be flushes in queue that aren't being run because of delayed flushes. There's no way to know from the metrics how many flushes are for each region, how many are delayed. Etc. > We should add either more metrics around this ( num per region, max per region, min per region ) or add on a UI page that has the list of compactions and flushes. > Or both. -- This message was sent by Atlassian JIRA (v6.4.14#64029)