beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinoth Chandar <vin...@uber.com>
Subject Re: KafkaIO()/Kafka Data Monitoring
Date Fri, 28 Oct 2016 00:26:48 GMT
I assume Flink uses the Kafka Simple Consumer. So, the built in JMX metrics
should be good enough?
https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Consumerstats

You can use something like https://github.com/jmxtrans/jmxtrans  to report
them to graphite et al.

My 2c,
Vinoth

On Thu, Oct 27, 2016 at 4:38 PM, amir bahmanyari <amirtousa@yahoo.com>
wrote:

> Hi Colleagues,
> Any suggestions on how to monitor the data rate coming to Beam app
> (KafkaIO()) running in a Flink cluster from Kafka?
> The Flink dashboard doesn't show any changes...its 0's all the time..
> I thought there may be a utility/tool that I can intercept/monitor the
> rate at which Kafka sends to the Beam app in Flink cluster?
>
> Thanks+regards.
>

Mime
View raw message