flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eswar Reddy <eswarareddyad...@gmail.com>
Subject Custom(application) Metrics - Piggyback on Flink's metrics infra or not?
Date Tue, 20 Sep 2016 06:29:55 GMT
Hi,

I see Flink support's built-in metrics to monitor various components of
Flink. In addition, one can register application specific(custom) metrics
to Flink's built-in metrics infra. The problem with this is user has to
develop his custom metrics using Flink's metrics framework/API rather than
a generic framework such as dropwizard. Alternatively, user can follow this
<http://www.michael-noll.com/blog/2013/11/06/sending-metrics-from-storm-to-graphite/#high-level-approach>
approach where his   dropwizard metrics push code is co-located with actual
app code within each Task and metrics are directly pushed to a backend
writer(say, Graphite) from each Task.

In this alternative, I am aware of having to handle mapping spatial
granularity of Flink's run-time with metrics namespace, but doing it myself
should not a big effort. Fault-tolerance comes automatically since app code
and metrics push code are co-located in the Task. Is there anything else
Flink's metrics infra handles automatically? Based on this I'd weigh using
good old dropwizard vs Flink specific metrics framework.

Finally, I guess feasibility an automatic dropwizard-to-flinkmetrics
translation utility can be checked out, but I would like to first
understand additional benefits of using flink's infra for custom metrics.

Thanks,
Eswar.

Mime
View raw message