phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karan Mehta (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3655) Metrics for PQS
Date Tue, 24 Jul 2018 20:57:00 GMT


Karan Mehta commented on PHOENIX-3655:

[~elserj] Getting back on this one after a long time. Need some more clarification before
I put up the implementation patch.

We have two phases of the patch, first is metric objects and second is exporting those objects.

Approach for first part, we need to use hbase-metrics and hbase-metrics-api modules (and the
metrics offered by them). These includes all classes that implement this {{org.apache.hadoop.hbase.metrics.Metric}}

We need to create a new {{org.apache.hadoop.hbase.metrics.MetricRegistryInfo}} and use the
static method {{;}} to get the corresponding {{org.apache.hadoop.hbase.metrics.MetricRegistry}}.
This registry object allows us to register new metrics, which in our case, will be conversion
of {{org.apache.phoenix.monitoring.GlobalMetric}} to {{org.apache.hadoop.hbase.metrics.Gauge}}.

Approach for second part, this is the part where we need to define the logic that metrics
from the registry defined above is pushed to JMX (through Hadoop Metrics 2, the default reporter)
and configure others if required. {{HBaseMetrics2HadoopMetricsAdapter}} class is supposed
to help us with that, since it converts hbase-metrics to hadoop-metrics. This adapter is an
implementation of MetricsSource class from hadoop2-metrics. We need to register it with {{DefaultMetricsSystem}}.
This happens as a part of the thread that {{GlobalMetricRegistriesAdapter}} runs every 10
seconds. The initialization for this one happens as a part of {{BaseSourceImpl}} constructor,
which is the class we want to move away from, if I understand correctly. Whats the best approach
then? Should I manually register the adapter with Hadoop's DefaultMetricsSystem? 

Let me know if this is not clear.

[~apurtell] [~xucang] would appreciate your insights as well.

> Metrics for PQS
> ---------------
>                 Key: PHOENIX-3655
>                 URL:
>             Project: Phoenix
>          Issue Type: New Feature
>    Affects Versions: 4.8.0
>         Environment: Linux 3.13.0-107-generic kernel, v4.9.0-HBase-0.98
>            Reporter: Rahul Shrivastava
>            Assignee: Karan Mehta
>            Priority: Major
>             Fix For: 4.15.0
>         Attachments: MetricsforPhoenixQueryServerPQS.pdf
>   Original Estimate: 240h
>  Remaining Estimate: 240h
> Phoenix Query Server runs a separate process compared to its thin client. Metrics collection
is currently done by i.e. at Phoenix driver level. We need the following
> 1. For every jdbc statement/prepared statement/ run by PQS , we need capability to collect
metrics at PQS level and push the data to external sink i.e. file, JMX , other external custom
> 2. Besides this global metrics could be periodically collected and pushed to the sink.

> 2. PQS can be configured to turn on metrics collection and type of collect ( runtime
or global) via hbase-site.xml
> 3. Sink could be configured via an interface in hbase-site.xml. 
> All metrics definition

This message was sent by Atlassian JIRA

View raw message