flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Way to get accumulators values *during* job execution ?
Date Wed, 02 Dec 2015 17:07:12 GMT
Hi Arnaud!

One thing you can do is to periodically retrieve them by querying the
monitoring API:

A nice approach would be to let the JobManager eagerly publish the
metrics.  I think that Christian Kreizfeld was working on an extension that
periodically publishes the accumulators to some metrics store. Maybe he can
comment on that.


On Wed, Dec 2, 2015 at 5:21 PM, LINZ, Arnaud <ALINZ@bouyguestelecom.fr>

> Hello,
> I use Grafana/Graphite to monitor my applications. The Flink GUI is really
> nice, but it disappears after the job completes and consequently is not
> suitable to long-term monitoring.
> For batch applications, I simply send the accumulator’s values at the end
> of the job to my Graphite base.
> For streaming applications, it’s more complex as the job never ends. It
> would be nice to have a way of getting current accumulator values (like in
> the GUI)  to push it periodically to Graphite in a monitoring thread. Is
> there any API to get the values during execution ?
> Best regards,
> Arnaud
> ------------------------------
> L'intégrité de ce message n'étant pas assurée sur internet, la société
> expéditrice ne peut être tenue responsable de son contenu ni de ses pièces
> jointes. Toute utilisation ou diffusion non autorisée est interdite. Si
> vous n'êtes pas destinataire de ce message, merci de le détruire et
> d'avertir l'expéditeur.
> The integrity of this message cannot be guaranteed on the Internet. The
> company that sent this message cannot therefore be held liable for its
> content nor attachments. Any unauthorized use or dissemination is
> prohibited. If you are not the intended recipient of this message, then
> please delete it and notify the sender.

View raw message