kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vahid S Hashemian" <vahidhashem...@us.ibm.com>
Subject Re: [DISCUSS] KIP-325: Extend Consumer Group Command to Show Beginning Offsets
Date Thu, 28 Jun 2018 20:05:12 GMT
Hi James,

Thanks for the feedback. I updated the KIP and added some of the benefits 
of this improvement (including some that you mentioned).

Regards.
--Vahid



From:   James Cheng <wushujames@gmail.com>
To:     dev@kafka.apache.org
Date:   06/27/2018 09:13 PM
Subject:        Re: [DISCUSS] KIP-325: Extend Consumer Group Command to 
Show Beginning Offsets



The “Motivation” section of the KIP says that the starting offset will be 
useful but doesn’t say why. Can you add a use-case or two to describe how 
it will be useful?

In our case (see 
https://github.com/wushujames/kafka-utilities/blob/master/ConsumerGroupLag/README.md
), we found the starting offset useful so that we could calculate 
partition size so that we could identify empty partitions (partitions 
where all the data had expired). In particular, we needed that info so 
that we could calculate “lag”. Consider that case where we are asked to 
mirror an abandoned topic where startOffset==endOffset==1000000. We would 
have no committed offset on it, and the topic has no data in it, so we 
won’t soon get any committed offset, and so “lag” is kind of undefined. We 
used the additional startOffset to allow us to detect this case.

-James

Sent from my iPhone

> On Jun 26, 2018, at 11:23 AM, Vahid S Hashemian 
<vahidhashemian@us.ibm.com> wrote:
> 
> Hi everyone,
> 
> I have created a trivial KIP to improve the offset reporting of the 
> consumer group command: 
> 
https://cwiki.apache.org/confluence/display/KAFKA/KIP-325%3A+Extend+Consumer+Group+Command+to+Show+Beginning+Offsets

> Looking forward to your feedback!
> 
> Thanks.
> --Vahid
> 
> 





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message