ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Sen" <d...@hortonworks.com>
Subject Re: Review Request 40650: Allow sending metrics as an async op with timeout and cancel
Date Tue, 24 Nov 2015 16:09:34 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40650/
-----------------------------------------------------------

(Updated Ноя. 24, 2015, 4:09 п.п.)


Review request for Ambari, Sumit Mohanty and Sid Wagle.


Repository: ambari


Description
-------

Noticed that there are no HDFS / YARN metrics sent to AMS.
HBase and host metrics are all ok.
There was a time where due to lack of resources AMS HBase was not keeping up with the writes
and customer had to reconfigure to get it to work.
Although after restarting AMS only the host and HBase metrics start showing. Note: Historical
AMS data still exists for HDFS and YARN, which means the Sinks were operational for some time.


Diffs
-----

  ambari-metrics/ambari-metrics-common/pom.xml 31c6b2e 
  ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
5aba016 
  ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
4f9b93e 
  ambari-metrics/ambari-metrics-flume-sink/src/test/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSinkTest.java
647e026 
  ambari-metrics/ambari-metrics-hadoop-sink/pom.xml ae821bf 
  ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
2d171d9 
  ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
dddbbd0 
  ambari-metrics/ambari-metrics-kafka-sink/pom.xml 15230aa 
  ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
dd7604b 
  ambari-metrics/ambari-metrics-kafka-sink/src/test/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporterTest.java
70f4850 
  ambari-metrics/ambari-metrics-storm-sink/pom.xml 7446167 
  ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
3a49e0a 
  ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
a0600e5 

Diff: https://reviews.apache.org/r/40650/diff/


Testing (updated)
-------

It was tested e2e:
- implemented delay of 60 second in AMS collector for all POST request
- deployed cluster with kafka, storm, flume, yarn, hdfs
- no metric graphs shown, no data in AMS. namenode.log contains
     2015-11-24 12:28:56,481 WARN  timeline.HadoopTimelineMetricsSink (HadoopTimelineMetricsSink.java:putMetrics(214))
- Unable to send metrics to collector by    address:http://c6404.ambari.apache.org:6188/ws/v1/timeline/metrics
- stop AMS only,
- replace patched AMS collector with AMS collector from the repo
- start AMS
- all graphs are shown without any service restart

Unit tests passed

[INFO] Reactor Summary:
[INFO] 
[INFO] ambari-metrics .................................... SUCCESS [1.490s]
[INFO] Ambari Metrics Common ............................. SUCCESS [2.802s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [5.549s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [2.472s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [4.217s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [3.127s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [23:17.034s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [3.129s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [4:30.237s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 28:10.394s
[INFO] Finished at: Tue Nov 24 18:09:05 EET 2015
[INFO] Final Memory: 102M/1155M


Thanks,

Dmytro Sen


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message