ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sid Wagle <swa...@hortonworks.com>
Subject Re: Review Request 43955: OOM on TimelineMetricCache in Nodemanager
Date Wed, 24 Feb 2016 22:25:16 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43955/#review120591
-----------------------------------------------------------



Patch looks good for OOM.
Did you look into possiblity of java.util.ConcurrentModificationException
>From the Jira it seems we might have a problem on update.

- Sid Wagle


On Feb. 24, 2016, 6:24 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43955/
> -----------------------------------------------------------
> 
> (Updated Feb. 24, 2016, 6:24 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-15100
>     https://issues.apache.org/jira/browse/AMBARI-15100
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Reported on ambari user group:
> 
> 2016-02-16 18:04:24,510 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373))
- java.util.ConcurrentModificationException
> 2016-02-16 18:04:34,509 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373))
- java.util.ConcurrentModificationException
> 2016-02-16 18:04:44,510 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373))
- java.util.ConcurrentModificationException
> 2016-02-16 18:04:59,424 FATAL yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(51))
- Thread Thread[timeline,5,main] threw an Error.  Shutting down now...
> java.lang.OutOfMemoryError: Java heap space
>        at java.util.HashMap.resize(HashMap.java:703)
>        at java.util.HashMap.putVal(HashMap.java:662)
>        at java.util.HashMap.put(HashMap.java:611)
>        at org.apache.hadoop.metrics2.sink.timeline.cache.TimelineMetricsCache$TimelineMetricHolder.put(TimelineMetricsCache.java:123)
>        at org.apache.hadoop.metrics2.sink.timeline.cache.TimelineMetricsCache.putTimelineMetric(TimelineMetricsCache.java:154)
>        at org.apache.hadoop.metrics2.sink.timeline.cache.TimelineMetricsCache.putTimelineMetric(TimelineMetricsCache.java:177)
>        at org.apache.hadoop.metrics2.sink.timeline.HadoopTimelineMetricsSink.putMetrics(HadoopTimelineMetricsSink.java:195)
>        at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.consume(MetricsSinkAdapter.java:186)
>        at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.consume(MetricsSinkAdapter.java:43)
>        at org.apache.hadoop.metrics2.impl.SinkQueue.consumeAll(SinkQueue.java:87)
>        at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.publishMetricsFromQueue(MetricsSinkAdapter.java:134)
>        at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter$1.run(MetricsSinkAdapter.java:88)
> 2016-02-16 18:04:59,433 INFO  util.ExitUtil (ExitUtil.java:halt(147)) - Halt with status
-1 Message: HaltException
> 
> 
> Diffs
> -----
> 
>   ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
15bd5f4 
> 
> Diff: https://reviews.apache.org/r/43955/diff/
> 
> 
> Testing
> -------
> 
> Unit tests passed
> 
> mvn clean test
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] ambari-metrics .................................... SUCCESS [0.845s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [2.570s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [4.165s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [2.147s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [2.900s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [0.472s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [4:40.041s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [1.116s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [6.003s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [34.107s]
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message