ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aravindan Vijayan" <avija...@hortonworks.com>
Subject Re: Review Request 39536: Ambari Server JVM crashed after several clicks in Web UI to navigate graph timerange
Date Thu, 22 Oct 2015 17:52:27 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39536/#review103617
-----------------------------------------------------------

Ship it!


Ship It!

- Aravindan Vijayan


On Oct. 22, 2015, 1:46 a.m., Sid Wagle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39536/
> -----------------------------------------------------------
> 
> (Updated Oct. 22, 2015, 1:46 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley, and Sumit
Mohanty.
> 
> 
> Bugs: AMBARI-13517
>     https://issues.apache.org/jira/browse/AMBARI-13517
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> - Max Heap restriction problem: Ehcache library that we use has a limit of how far in
the object graph it traverses to find size of an Cached reference, this is to make it performant.
Default limit = 1000.
> - User gets following Warning which are harmless unless the data cannot fit in memory
and eviction does not kick in:
> 
> WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000 object references
was reached while attempting to calculate the size of the object graph. Severe performance
degradation could occur 
> if the sizing operation continues. This can be avoided by setting the CacheManger or
Cache <sizeOfPolicy> elements maxDepthExceededBehavior to "abort" or adding stop points
with @IgnoreSizeOf annotations. If performance degradation 
> is NOT an issue at the configured limit, raise the limit value using the CacheManager
or Cache <sizeOfPolicy> elements maxDepth attribute. For more information, see the Ehcache
configuration documentation.
> 
> _Objective of the patch_:
> - Provide a custom Sizing engine for Ehcache that will provide a close approximation
of the data in the cache with significant perf gain. Premise: Since we know the data structures
we can make better estimates.
> - Expectation from sizing engine : Discrepancy of less than 10K for a DS of size 10 MB
(Proven with a unit test)
> 
> 
> Diffs
> -----
> 
>   ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java
8b8df06 
>   ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
224b490 
>   ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java
4a13d63 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
9c0b94d 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
9488316 
>   ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java
573e09d 
>   ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java
b7b1737 
>   ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java
a3640d0 
>   ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java
26771d7 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java
522c0bd 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java
6d80687 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java
PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java
c30c5eb 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java
PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java
3432e1d 
> 
> Diff: https://reviews.apache.org/r/39536/diff/
> 
> 
> Testing
> -------
> 
> Relevant unit test pass.
> ambari-server:
>  AMSPropertyProviderTest
>  AMSReportPropertyProviderTest
>  StackDefinedPropertyProviderTest
>  TimelineMetricsCacheTest
>  TimleineMetricsCacheSizingTest
> ambari-metrics:
>  All test passed
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message