ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 38070: Optimize aggregator queries by performing GROUP BY on server
Date Thu, 03 Sep 2015 00:39:56 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38070/#review97578
-----------------------------------------------------------



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
(line 67)
<https://reviews.apache.org/r/38070/#comment153588>

    Why would initializeSubsystem be called from multiple threads?



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
(line 228)
<https://reviews.apache.org/r/38070/#comment153590>

    safer to put a try catch, or tryParseInt



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java
(line 26)
<https://reviews.apache.org/r/38070/#comment153591>

    Switch this to fully qualified import.



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
(line 237)
<https://reviews.apache.org/r/38070/#comment153596>

    In general, add more documentation to the classes and functions so it's easier to follow.



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
(line 242)
<https://reviews.apache.org/r/38070/#comment153594>

    If SERVER_TIME is likely to be unique each time, then is UPSERT really needed, or can
this just be an INSERT?


- Alejandro Fernandez


On Sept. 2, 2015, 10:17 p.m., Sid Wagle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38070/
> -----------------------------------------------------------
> 
> (Updated Sept. 2, 2015, 10:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Mahadev Konar, Sumit Mohanty,
and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-12983
>     https://issues.apache.org/jira/browse/AMBARI-12983
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Use GroupBy aggregators wherever possible.
> 
> Query plan:
> 
> 0: jdbc:phoenix:localhost:61181:/hbase> explain select SUM(METRIC_SUM), SUM(METRIC_COUNT),
MAX(METRIC_MAX), MIN(METRIC_MIN), METRIC_NAME, HOSTNAME, APP_ID from METRIC_RECORD WHERE SERVER_TIME
> 1440106830000 AND SERVER_TIME < 1440106950000 GROUP BY METRIC_NAME, HOSTNAME, APP_ID;
> +------------+
> |    PLAN    |
> +------------+
> | CLIENT PARALLEL 1-WAY FULL SCAN OVER METRIC_RECORD |
> |     SERVER FILTER BY (SERVER_TIME > 1440106830000 AND SERVER_TIME < 1440106950000)
|
> |     SERVER AGGREGATE INTO DISTINCT ROWS BY [METRIC_NAME, HOSTNAME, APP_ID] |
> | CLIENT MERGE SORT |
> +------------+
> 
> Scan depends on number of Regions, so it would by lets say 4 way parallel scan with 4
Regions.
> Next effort is to come up with a good split policy for the precision table.
> 
> 
> Diffs
> -----
> 
>   ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector bdf00b5

>   ambari-metrics/ambari-metrics-timelineservice/pom.xml d994d52 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryClientService.java
8a37a57 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
c615804 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
d4f919e 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
37fb088 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java
642fcfe 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
PRE-CREATION 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
PRE-CREATION 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/Condition.java
e0cb3d0 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java
462c8d9 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/EmptyCondition.java
PRE-CREATION 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
ef0f4ce 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/SplitByMetricNamesCondition.java
c8b8709 
>   ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java
13fa348 
>   ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java
b480b7a 
>   ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
06e153b 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
ef6fc58 
>   ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml
2d11412 
>   ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
c716bea 
> 
> Diff: https://reviews.apache.org/r/38070/diff/
> 
> 
> Testing
> -------
> 
> All ams unit test pass.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message