From issues-return-342172-archive-asf-public=cust-asf.ponee.io@hbase.apache.org Thu Apr 5 17:42:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 3F2FB18063B for ; Thu, 5 Apr 2018 17:42:04 +0200 (CEST) Received: (qmail 6065 invoked by uid 500); 5 Apr 2018 15:42:02 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 6050 invoked by uid 99); 5 Apr 2018 15:42:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Apr 2018 15:42:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id E9C00C0040 for ; Thu, 5 Apr 2018 15:42:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.311 X-Spam-Level: X-Spam-Status: No, score=-110.311 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 8h-zJ--Ch9zz for ; Thu, 5 Apr 2018 15:42:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 0D2015F23D for ; Thu, 5 Apr 2018 15:42:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 95265E0842 for ; Thu, 5 Apr 2018 15:42:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 57DF325622 for ; Thu, 5 Apr 2018 15:42:00 +0000 (UTC) Date: Thu, 5 Apr 2018 15:42:00 +0000 (UTC) From: "stack (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-20234) Expose in-memory compaction metrics MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-20234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427136#comment-16427136 ] stack commented on HBASE-20234: ------------------------------- bq. There are currently no per-store counters that I have described (number of in-memory-flushes, number of saved bytes), nor that they make it up to region-level. No problem to add them. [~anastas] Thanks. Be careful on per-store counters. HBase is already crippled keeping counts (one of the main users of CPU). The per-region counters are already costly. If we were to do per-store, the number of counts would blossom. bq. Alternatively we can coordinate the collection of the in-memory-flush counters to the flush-to-disk update, that already exists in the code. If no flush to disk, no counters. If no flush to disk, then we are not doing heavy writes. Maybe its ok to do this. It'd be better than a background thread doing counter updates. When hbase is not doing counters, its context switching between all the various background threads that do caretaking; no cpu left over to do actual work (smile). Thanks for looking at this A. Keep asking questions. The metrics system is cryptic. Lets save you getting lost in it if we can help it. > Expose in-memory compaction metrics > ----------------------------------- > > Key: HBASE-20234 > URL: https://issues.apache.org/jira/browse/HBASE-20234 > Project: HBase > Issue Type: Sub-task > Reporter: stack > Assignee: Anastasia Braginsky > Priority: Major > > Hard to glean insight from how well in-memory compaction is doing currently. It dumps stats into the logs but better if they were available to a dashboard. This issue is about exposing a couple of helpful counts. There are already by-region metrics. We can add a few for in-memory compaction (Help me out [~anastas]... what counts would be best to expose). > Flush related metrics include.... > {code} > Namespace_default_table_tsdb-tree_region_cfbf23e7330a1a2bbde031f9583d3415_metric_flushesQueuedCount: { > description: "Number flushes requested/queued for this region", > value: 0 > { > description: "The number of cells flushed to disk", > value: 0 > }, > { > description: "The total amount of data flushed to disk, in bytes", > value: 0 > }, > ... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)