phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karan Mehta (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3655) Metrics for PQS
Date Thu, 17 May 2018 20:08:00 GMT


Karan Mehta commented on PHOENIX-3655:

{quote}The rest of what you were saying is about on track: we use the hbase-metrics-api to
construct the things we're measuring, sending them to the MetricsRegistry. Then, HBase (or
the reporter we configure on the Registry) would take care of pushing those to Hadoop Metrics2
sink. There may be something in place already with the underlying dropwizard metrics implementation
to push all of these to JMX (as an aside).
[~elserj] The stuff you are talking about is implemented as a part of hbase-hadoop2-compat
library on which phoenix-core depends. I would prefer re-using the code that provides the
{{HBaseMetrics2HadoopMetricsAdapter}} for converting the metrics. Also, {{GlobalMetricRegistriesAdapter}}
class helps me directly dump the hbase registry to JMX.
{quote}I fear you might be getting sucked into some old metrics cruft in HBase, [~karanmehta93].
BaseSourceImpl and other classes in hbase-hadoop2-compat are vestigial to prevent having to
rewrite all of hbase-server to use the new hbase-metrics-api. I would think that if you're
coming in here fresh, you could just use hbase-metrics-api only and ignore all of that other
 Agreed to some extent, please refer to my old comment and advise how we can proceed.
{quote}GLOBAL_MUTATION_BYTES is useful for trending, to see how the amount of data written
by a client changes over time
This is *NOT* per client, be advised. [~tdsilva] This number will keep growing, we would ideally
need the rate at which it is growing. If its not the peak hours of day, the number of mutations
will be less and vice versa. A Guage to track the raw value might not be super useful here.
HBase-metrics Meter is a DropwizardMeter and provides the rate of change in last 1 min, 5
min and 15 mins, might be useful here (though not sure).
{quote}Are you planning on only converting GlobalClientMetrics to use hbase metrics? Or are
you going to change MutationMetricQueue, OverAllQueryMetrics etc as well?
Only the GlobalClientMetrics for this Jira. The current metrics implementations are simple
counters as such internally, If we plan to expose them as hbase counters, the work should
be trivial. However if we decide to model them as other types, more work might be involved.
I don't want to change how the metrics are exported from various queues.

Also, these metrics keep a track of number of samples of the metric. How can that be used
for our case? I could not find any Metric that uses it.

[~apurtell] thoughts?

> Metrics for PQS
> ---------------
>                 Key: PHOENIX-3655
>                 URL:
>             Project: Phoenix
>          Issue Type: New Feature
>    Affects Versions: 4.8.0
>         Environment: Linux 3.13.0-107-generic kernel, v4.9.0-HBase-0.98
>            Reporter: Rahul Shrivastava
>            Assignee: Karan Mehta
>            Priority: Major
>             Fix For: 4.15.0
>         Attachments: MetricsforPhoenixQueryServerPQS.pdf
>   Original Estimate: 240h
>  Remaining Estimate: 240h
> Phoenix Query Server runs a separate process compared to its thin client. Metrics collection
is currently done by i.e. at Phoenix driver level. We need the following
> 1. For every jdbc statement/prepared statement/ run by PQS , we need capability to collect
metrics at PQS level and push the data to external sink i.e. file, JMX , other external custom
> 2. Besides this global metrics could be periodically collected and pushed to the sink.

> 2. PQS can be configured to turn on metrics collection and type of collect ( runtime
or global) via hbase-site.xml
> 3. Sink could be configured via an interface in hbase-site.xml. 
> All metrics definition

This message was sent by Atlassian JIRA

View raw message