ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LaStrange, Adam" <Adam.LaStra...@sig.com>
Subject Yarn nodemanager java.lang.OutOfMemory
Date Thu, 18 Feb 2016 18:52:00 GMT
I have seen several instances of the Yarn nodemanager shutting down due to this exception:

2016-02-16 18:04:24,510 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373)) - java.util.ConcurrentModificationException
2016-02-16 18:04:34,509 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373)) - java.util.ConcurrentModificationException
2016-02-16 18:04:44,510 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:run(373)) - java.util.ConcurrentModificationException
2016-02-16 18:04:59,424 FATAL yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(51))
- Thread Thread[timeline,5,main] threw an Error.  Shutting down now...
java.lang.OutOfMemoryError: Java heap space
       at java.util.HashMap.resize(HashMap.java:703)
       at java.util.HashMap.putVal(HashMap.java:662)
       at java.util.HashMap.put(HashMap.java:611)
       at org.apache.hadoop.metrics2.sink.timeline.cache.TimelineMetricsCache$TimelineMetricHolder.put(TimelineMetricsCache.java:123)
       at org.apache.hadoop.metrics2.sink.timeline.cache.TimelineMetricsCache.putTimelineMetric(TimelineMetricsCache.java:154)
       at org.apache.hadoop.metrics2.sink.timeline.cache.TimelineMetricsCache.putTimelineMetric(TimelineMetricsCache.java:177)
       at org.apache.hadoop.metrics2.sink.timeline.HadoopTimelineMetricsSink.putMetrics(HadoopTimelineMetricsSink.java:195)
       at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.consume(MetricsSinkAdapter.java:186)
       at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.consume(MetricsSinkAdapter.java:43)
       at org.apache.hadoop.metrics2.impl.SinkQueue.consumeAll(SinkQueue.java:87)
       at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.publishMetricsFromQueue(MetricsSinkAdapter.java:134)
       at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter$1.run(MetricsSinkAdapter.java:88)
2016-02-16 18:04:59,433 INFO  util.ExitUtil (ExitUtil.java:halt(147)) - Halt with status -1
Message: HaltException


This code is located in the ambari-metrics project.  Has anyone seen this before or have advice
on how to proceed?  Running with HDP-2.3.0.0-2557, Ambari Metrics 0.1.0, YARN 2.7.1.2.3. 
Thanks

-Adam L

________________________________

IMPORTANT: The information contained in this email and/or its attachments is confidential.
If you are not the intended recipient, please notify the sender immediately by reply and immediately
delete this message and all its attachments. Any review, use, reproduction, disclosure or
dissemination of this message or any attachment by an unintended recipient is strictly prohibited.
Neither this message nor any attachment is intended as or should be construed as an offer,
solicitation or recommendation to buy or sell any security or other financial instrument.
Neither the sender, his or her employer nor any of their respective affiliates makes any warranties
as to the completeness or accuracy of any of the information contained herein or that this
message or any of its attachments is free of viruses.

Mime
View raw message