flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Yammer metrics
Date Fri, 08 Apr 2016 11:05:01 GMT
I'm very much aware of how Yammer works.

As the slides you linked show (near the end) is that there are several 
small issues with the general-purpose reporters offered by yammer.

Instead of hacking around those issues i would very much prefer creating 
our own reporters that are, again, /based/ on the yammer reporters as 
the concept is sound, but can properly interact with the rest of the system.

Long-term this will create cleaner code, easier debugging and allow us 
to adjust things as required at any time.

On 08.04.2016 12:39, Sanne de Roever wrote:
> I forgot to add some extra information, still all tentative.
> Earlier (erm, 14 years ago to be honest), I also worked on a JMX 
> monitoring system, and it turned out to be a pain to identify all the 
> components, write the correct jmx queries and then to plot everything. 
> It was hard. This presentation echos this sentiment, and proposes the 
> Yammer route: 
> http://www.slideshare.net/NaderGan/cassandra-jmxexpresshow-do-we-monitor-cassandra-using-graphite-leveraging-yammer-codahale-library

> On Fri, Apr 8, 2016 at 12:12 PM, Sanne de Roever 
> <sanne.de.roever@gmail.com <mailto:sanne.de.roever@gmail.com>> wrote:
>     Thanks Chesnay.
>     Might I make a tentative case for Yammer? I'm not an expert, but I
>     am currently trying to pull together information on this and was
>     reviewing jmxtrans. This is all tentative, I've just dived in a
>     few days ago. Please find the information below.
>     Using Yammer it is possible to load an MBean that queries all
>     metrics and exports them en masse to a target destination, statsd
>     or graphite for example, but any destination will to. The gist is
>     that the metrics do not have to be queried one by one, and that
>     the export can be arranged by loading an extra jar.
>     An example of this setup can be seen at:
>     https://github.com/airbnb/kafka-statsd-metrics2
>     Put the MBean jar in the classpath, add a config file, and presto.
>     This would work for Kafka, and Cassandra, and I would not need a
>     separate JMXtrans server.
>     On Fri, Apr 8, 2016 at 11:01 AM, Chesnay Schepler
>     <chesnay@apache.org <mailto:chesnay@apache.org>> wrote:
>         Flink currently doesn't expose any metrics beyond those shown
>         in the Dashboard.
>         I am currently working on integrating a new metrics system
>         that is partly /based/ on Yammer/Codahale/Dropwizard metrics.
>         For a first version it is planned to export metrics only via
>         JMX as this effectively covers all use-cases with the help of
>         JMXTrans and similar tools.
>         Reporting to specific systems is something we want to add as
>         well though.
>         Regards,
>         Chesnay Schepler
>         On 08.04.2016 09:32, Sanne de Roever wrote:
>>         Hi,
>>         I´m looking into setting up monitoring for our (Flink)
>>         environment and realized that both Kafka and Cassandra use
>>         the yammer metrics library. This library enables the direct
>>         export of all metrics to Graphite (and possibly statsd). Does
>>         Flink use Yammer metrics?
>>         Cheers,
>>         Sanne

View raw message