Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5F91918EB4 for ; Mon, 5 Oct 2015 20:50:42 +0000 (UTC) Received: (qmail 62594 invoked by uid 500); 5 Oct 2015 20:50:27 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 62548 invoked by uid 500); 5 Oct 2015 20:50:27 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 62537 invoked by uid 99); 5 Oct 2015 20:50:27 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Oct 2015 20:50:27 +0000 Date: Mon, 5 Oct 2015 20:50:27 +0000 (UTC) From: "Nick Dimiduk (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-12911) Client-side metrics MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944004#comment-14944004 ] Nick Dimiduk commented on HBASE-12911: -------------------------------------- Woo, green build. What are your thoughts on the latest patch [~stack], [~eclark]? You guys okay with the approach to pb unpacking as a perf optimization? The asserts should allow us to notice changes in protobuf internals at test time at least. > Client-side metrics > ------------------- > > Key: HBASE-12911 > URL: https://issues.apache.org/jira/browse/HBASE-12911 > Project: HBase > Issue Type: New Feature > Components: Client, Operability, Performance > Reporter: Nick Dimiduk > Assignee: Nick Dimiduk > Fix For: 2.0.0, 1.3.0 > > Attachments: 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 12911-0.98.00.patch, 12911-branch-1.00.patch, 12911.yammer.jpg, 12911.yammer.v00.patch, 12911.yammer.v01.patch, 12911.yammer.v02.patch, 12911.yammer.v02.patch, 12911.yammer.v03.patch, 12911.yammer.v03.patch, am.jpg, client metrics RS-Master.jpg, client metrics client.jpg, conn_agg.jpg, connection attributes.jpg, ltt.jpg, standalone.jpg > > > There's very little visibility into the hbase client. Folks who care to add some kind of metrics collection end up wrapping Table method invocations with {{System.currentTimeMillis()}}. For a crude example of this, have a look at what I did in {{PerformanceEvaluation}} for exposing requests latencies up to {{IntegrationTestRegionReplicaPerf}}. The client is quite complex, there's a lot going on under the hood that is impossible to see right now without a profiler. Being a crucial part of the performance of this distributed system, we should have deeper visibility into the client's function. > I'm not sure that wiring into the hadoop metrics system is the right choice because the client is often embedded as a library in a user's application. We should have integration with our metrics tools so that, i.e., a client embedded in a coprocessor can report metrics through the usual RS channels, or a client used in a MR job can do the same. > I would propose an interface-based system with pluggable implementations. Out of the box we'd include a hadoop-metrics implementation and one other, possibly [dropwizard/metrics|https://github.com/dropwizard/metrics]. > Thoughts? -- This message was sent by Atlassian JIRA (v6.3.4#6332)