kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Davey <jon...@gmail.com>
Subject Measuring remaining data
Date Thu, 02 Jan 2014 21:15:02 GMT
I need to provide a report that shows the total amount of unread data
available to a consumer group for a particular set of topics.

Right now I do this by adding custom offset tracking logic to the
producers and consumers. This works but I think I can do better with
one of the following ideas:

* Running something alongside each broker that grabs the data from the
log dir and zookeeper
* Extending Kafka to do the same as above but expose it as a request
through the protocol

If I wanted to do these things, where would be a good place to start looking?

View raw message