ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Wagle (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMBARI-12983) Optimize aggregator queries by performing GROUP BY on server
Date Wed, 02 Sep 2015 19:39:45 GMT
Siddharth Wagle created AMBARI-12983:
----------------------------------------

             Summary: Optimize aggregator queries by performing GROUP BY on server
                 Key: AMBARI-12983
                 URL: https://issues.apache.org/jira/browse/AMBARI-12983
             Project: Ambari
          Issue Type: Task
          Components: ambari-metrics
    Affects Versions: 2.1.1
            Reporter: Siddharth Wagle
            Assignee: Siddharth Wagle
            Priority: Critical
             Fix For: 2.1.2


Query to do calculate host level time aggregates.

*Sample*
{code}
select SUM(METRIC_SUM), SUM(METRIC_COUNT), MAX(METRIC_MAX), MIN(METRIC_MIN), METRIC_NAME,
HOSTNAME, APP_ID from METRIC_RECORD WHERE METRIC_NAME IN ('bytes_in','rpc.rpc.RpcAuthorizationSuccesses')
SERVER_TIME > 1440106830000 AND SERVER_TIME < 1440106950000 GROUP BY METRIC_NAME, HOSTNAME,
APP_ID limit 50;
0: jdbc:phoenix:localhost:61181:/hbase> select SUM(METRIC_SUM), SUM(METRIC_COUNT), MAX(METRIC_MAX),
MIN(METRIC_MIN), METRIC_NAME, HOSTNAME, APP_ID from METRIC_RECORD WHERE METRIC_NAME IN ('bytes_in','rpc.rpc.RpcAuthorizationSuccesses')
AND SERVER_TIME > 1440106830000 AND SERVER_TIME < 1440106950000 GROUP BY METRIC_NAME,
HOSTNAME, APP_ID limit 50;

+-----------------+-------------------+-----------------+-----------------+-------------+------------+------------+
| SUM(METRIC_SUM) | SUM(METRIC_COUNT) | MAX(METRIC_MAX) | MIN(METRIC_MIN) | METRIC_NAME |
 HOSTNAME  |   APP_ID   |
+-----------------+-------------------+-----------------+-----------------+-------------+------------+------------+
| 1546242.5521201263 | 24                | 564689.2744108599 | 616.7147803365921 | bytes_in
   | ams-test-1.c.pramod-thangali.internal | HOST       |
| 1521023.3303401857 | 24                | 825825.3247578086 | 220.66636815664341 | bytes_in
   | ams-test-2.c.pramod-thangali.internal | HOST       |
| 153685.92452883872 | 24                | 25389.11630078307 | 448.15683323834327 | bytes_in
   | ams-test-3.c.pramod-thangali.internal | HOST       |
| 0.0             | 14                | 0.0             | 0.0             | rpc.rpc.RpcAuthorizationSuccesses
| ams-test-1.c.pramod-thangali.internal | datanode   |
| 17.0            | 14                | 2.0             | 0.0             | rpc.rpc.RpcAuthorizationSuccesses
| ams-test-1.c.pramod-thangali.internal | namenode   |
| 0.0             | 14                | 0.0             | 0.0             | rpc.rpc.RpcAuthorizationSuccesses
| ams-test-2.c.pramod-thangali.internal | datanode   |
+-----------------+-------------------+-----------------+-----------------+-------------+------------+------------+
{code}

*Query plan*:
{code}
0: jdbc:phoenix:localhost:61181:/hbase> explain select SUM(METRIC_SUM), SUM(METRIC_COUNT),
MAX(METRIC_MAX), MIN(METRIC_MIN), METRIC_NAME, HOSTNAME, APP_ID from METRIC_RECORD WHERE SERVER_TIME
> 1440106830000 AND SERVER_TIME < 1440106950000 GROUP BY METRIC_NAME, HOSTNAME, APP_ID;
+------------+
|    PLAN    |
+------------+
| CLIENT PARALLEL 1-WAY FULL SCAN OVER METRIC_RECORD |
|     SERVER FILTER BY (SERVER_TIME > 1440106830000 AND SERVER_TIME < 1440106950000)
|
|     SERVER AGGREGATE INTO DISTINCT ROWS BY [METRIC_NAME, HOSTNAME, APP_ID] |
| CLIENT MERGE SORT |
+------------+
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message