ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aravindan Vijayan" <avija...@hortonworks.com>
Subject Re: Review Request 40650: Allow sending metrics as an async op with timeout and cancel
Date Tue, 24 Nov 2015 18:24:46 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40650/#review107812
-----------------------------------------------------------

Ship it!


Ship It!

- Aravindan Vijayan


On Nov. 24, 2015, 4:09 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40650/
> -----------------------------------------------------------
> 
> (Updated Nov. 24, 2015, 4:09 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty and Sid Wagle.
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Noticed that there are no HDFS / YARN metrics sent to AMS.
> HBase and host metrics are all ok.
> There was a time where due to lack of resources AMS HBase was not keeping up with the
writes and customer had to reconfigure to get it to work.
> Although after restarting AMS only the host and HBase metrics start showing. Note: Historical
AMS data still exists for HDFS and YARN, which means the Sinks were operational for some time.
> 
> 
> Diffs
> -----
> 
>   ambari-metrics/ambari-metrics-common/pom.xml 31c6b2e 
>   ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
5aba016 
>   ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
4f9b93e 
>   ambari-metrics/ambari-metrics-flume-sink/src/test/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSinkTest.java
647e026 
>   ambari-metrics/ambari-metrics-hadoop-sink/pom.xml ae821bf 
>   ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
2d171d9 
>   ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
dddbbd0 
>   ambari-metrics/ambari-metrics-kafka-sink/pom.xml 15230aa 
>   ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
dd7604b 
>   ambari-metrics/ambari-metrics-kafka-sink/src/test/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporterTest.java
70f4850 
>   ambari-metrics/ambari-metrics-storm-sink/pom.xml 7446167 
>   ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
3a49e0a 
>   ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
a0600e5 
> 
> Diff: https://reviews.apache.org/r/40650/diff/
> 
> 
> Testing
> -------
> 
> It was tested e2e:
> - implemented delay of 60 second in AMS collector for all POST request
> - deployed cluster with kafka, storm, flume, yarn, hdfs
> - no metric graphs shown, no data in AMS. namenode.log contains
>      2015-11-24 12:28:56,481 WARN  timeline.HadoopTimelineMetricsSink (HadoopTimelineMetricsSink.java:putMetrics(214))
- Unable to send metrics to collector by    address:http://c6404.ambari.apache.org:6188/ws/v1/timeline/metrics
> - stop AMS only,
> - replace patched AMS collector with AMS collector from the repo
> - start AMS
> - all graphs are shown without any service restart
> 
> Unit tests passed
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] ambari-metrics .................................... SUCCESS [1.490s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [2.802s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [5.549s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [2.472s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [4.217s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [3.127s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [23:17.034s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [3.129s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [4:30.237s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 28:10.394s
> [INFO] Finished at: Tue Nov 24 18:09:05 EET 2015
> [INFO] Final Memory: 102M/1155M
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message