samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milinda Pathirage <>
Subject Re: monitoring best practices
Date Fri, 26 Jun 2015 01:36:49 GMT
Hi Christopher,

Recently I did something similar but for getting performance numbers out
from Samza. I used InfluxDB. I wrote a stream task which consumes the
metrics topics and deployed it as a another Samza job. From that job I
pushed metrics in to InfluxDB.


On Thu, Jun 25, 2015 at 5:22 PM, Christopher Chamberlin <> wrote:

> I see in the Samza metrics documentation that there are two basic ways to
> get metrics from Samza to a metrics repository: 1) write a custom
> MetricsReporter to push the metrics directly, perhaps using an
> ExecutorService or similar to perform batching, or 2) consume the metrics
> Kafka queue and push them from there, letting the built-in
> MetricsSnapshotReporter do the batching.
> Can anyone running Samza in production provide any insight into which of
> these to prefer?
> I'm looking to get my metrics out to Prometheus, probably via a Pushgateway
> endpoint.
> I see the pending SAMZA-340 patch to add Graphite support using approach
> #1. I like going directly from the monitored container to the monitoring
> system (fewer moving parts than going via Kafka in method #2), but I'd
> rather not re-implement the batching and other logic in the existing
> SnapshotMetricsReporter.
> Thanks.

Milinda Pathirage

PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University

twitter: milindalakmal
skype: milinda.pathirage

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message