hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12911) Client-side metrics
Date Sat, 05 Sep 2015 05:29:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14731812#comment-14731812
] 

stack commented on HBASE-12911:
-------------------------------

Tried the patch local. I see the 'tag.' prefix on Context and Hostname too on Client and our
old beans. Weird. Whats that mess about?  Can't change it at this stage. And yeah, what are
all those extra connections about (in standalone I see four w/ just one of them carrying the
extra localhost-port bean).

So in standalone I see a Master, a RegionServer, and a Client under Hadoop.  Under client
are connection beans. I'd imagine I'm seeing the client hosted by the master and one hosted
by regionserver.... Hard to tell what connection is hosted where and where it is connected.
Is that fixable?

I like your picture of aggregation.

bg. So you thinking I should scrap the host-level data and just show aggregation for the connection?
Or just add aggregation to what I have? My thinking is that when you want to diagnose slow
queries, it helps to see individual host details broken out.

I was thinking that you'd want aggregation first on the connection. If slow, yeah, would be
nice to dig down to figure if a particular connection slower than others. If slow query though,
how you find it? You have to dig through each of these sub beans (though I suppose you would
not be using jconsole... you'd be grepping a json dump or text dump of the bean content...or
some such).





> Client-side metrics
> -------------------
>
>                 Key: HBASE-12911
>                 URL: https://issues.apache.org/jira/browse/HBASE-12911
>             Project: HBase
>          Issue Type: New Feature
>          Components: Client, Operability, Performance
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>             Fix For: 2.0.0, 1.3.0
>
>         Attachments: 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch,
0001-HBASE-12911-Client-side-metrics.patch, am.jpg, client metrics RS-Master.jpg, client metrics
client.jpg, conn_agg.jpg, connection attributes.jpg, ltt.jpg, standalone.jpg
>
>
> There's very little visibility into the hbase client. Folks who care to add some kind
of metrics collection end up wrapping Table method invocations with {{System.currentTimeMillis()}}.
For a crude example of this, have a look at what I did in {{PerformanceEvaluation}} for exposing
requests latencies up to {{IntegrationTestRegionReplicaPerf}}. The client is quite complex,
there's a lot going on under the hood that is impossible to see right now without a profiler.
Being a crucial part of the performance of this distributed system, we should have deeper
visibility into the client's function.
> I'm not sure that wiring into the hadoop metrics system is the right choice because the
client is often embedded as a library in a user's application. We should have integration
with our metrics tools so that, i.e., a client embedded in a coprocessor can report metrics
through the usual RS channels, or a client used in a MR job can do the same.
> I would propose an interface-based system with pluggable implementations. Out of the
box we'd include a hadoop-metrics implementation and one other, possibly [dropwizard/metrics|https://github.com/dropwizard/metrics].
> Thoughts?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message