kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Davey <jon...@gmail.com>
Subject Re: Measuring remaining data
Date Thu, 02 Jan 2014 22:26:58 GMT
That's certainly a step in the right direction. The thing is, I'd
ideally like to use bytes rather than just messages and that data no
longer seems to be available in Kafka 0.8. KAFKA-656 might give me
what I need but that seems abandoned.

On 2 January 2014 21:26, Guozhang Wang <wangguoz@gmail.com> wrote:
> Hi Jonathan,
>
> Have you checked out the ConsumerOffsetChecker tool?
>
> Guozhang
>
>
> On Thu, Jan 2, 2014 at 1:15 PM, Jonathan Davey <jond3k@gmail.com> wrote:
>
>> I need to provide a report that shows the total amount of unread data
>> available to a consumer group for a particular set of topics.
>>
>> Right now I do this by adding custom offset tracking logic to the
>> producers and consumers. This works but I think I can do better with
>> one of the following ideas:
>>
>> * Running something alongside each broker that grabs the data from the
>> log dir and zookeeper
>> * Extending Kafka to do the same as above but expose it as a request
>> through the protocol
>>
>> If I wanted to do these things, where would be a good place to start
>> looking?
>>
>
>
>
> --
> -- Guozhang

Mime
View raw message