ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sid Wagle" <swa...@hortonworks.com>
Subject Re: Review Request 37146: Create a Caching layer that provides sliding window behavior for metric requests to Ambari
Date Thu, 13 Aug 2015 04:03:14 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37146/
-----------------------------------------------------------

(Updated Aug. 13, 2015, 4:03 a.m.)


Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jaimin Jetly, Mahadev Konar, and
Sumit Mohanty.


Changes
-------

Mutiple changes based on actual test results:
- Added a 5 minute interval to cover for metrics never present on the latest time boundary.
- Additional unit tests added.
- New method to quietly update the cache key with new boundary info.


Bugs: AMBARI-12654
    https://issues.apache.org/jira/browse/AMBARI-12654


Repository: ambari


Description
-------

Initial review requeted.

Objectives:
-    In-memory caching of timeseries data for 1 hour queries.
-    Support of service and dashboard metrics only
-    Cache refresh is for a small segment of data vs requesting 1 hour data every time.
-    On demand cache with sliding window implementation
-    Zero/Null padding data does not appear in the cache
-    Cache functionality can be turned on / off using config knob
-    Caching policy is a cache-as-sor (Cache as System of Record), all queries for Service
Metrics will be returned from cache if caching is enabled and query window fits Cache timeseries
window, example 1 hour.
-    Host metric queries will still goto AMS, this prevents host specific data to be cached.
This decision is based on usage/access pattern from Ambari web UI.


Diffs (updated)
-----

  ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetrics.java
383079a 
  ambari-server/pom.xml 5725446 
  ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java 371d5d2

  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
17b6d4a 
  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
a7f206a 
  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java 5644ca5

  ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java e048ec5

  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
380a0fe 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackDefinedPropertyProvider.java
3a6d30b 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/TemporalInfoImpl.java
2ffe984 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPropertyProvider.java
9fa9ca4 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPropertyProviderProxy.java
57a8e7d 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsReportPropertyProvider.java
5399436 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsReportPropertyProviderProxy.java
a92cb37 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSComponentPropertyProvider.java
d5f415a 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSHostComponentPropertyProvider.java
8ee2acb 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSHostPropertyProvider.java
ca9d685 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
6667134 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSReportPropertyProvider.java
a095206 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/MetricsRequestHelper.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineAppMetricCacheKey.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCache.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheValue.java
PRE-CREATION 
  ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackDefinedPropertyProviderTest.java
b8e0596 
  ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java
c8007c8 
  ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSReportPropertyProviderTest.java
3ee64fa 
  ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java
PRE-CREATION 

Diff: https://reviews.apache.org/r/37146/diff/


Testing
-------

Unit tests pass. Manual testing in progress.


Thanks,

Sid Wagle


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message