kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Gustafson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-3059) ConsumerGroupCommand should allow resetting offsets for consumer groups
Date Tue, 05 Jan 2016 22:01:39 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083922#comment-15083922

Jason Gustafson commented on KAFKA-3059:

One problem is defining the behavior of these operations when the consumer group is still
active. The simple option would be to forbid them. Users would have to shutdown all consumers
in the group, which actually seems pretty reasonable. But if there are legitimate use cases
which call for overriding offsets while a group is still active, then it will take some tricky
coordination to get all group members to reload offsets.

As far as implementation, writing to the offsets topic directly does not seem viable since
there would be no way to verify whether the group is active or not. Even if we wanted to support
offset updates while the group is active, it would still require some coordination with the
group coordinator. That means we'll either need to override the commit API to support deletion
(e.g. by using -1 as the offset for each partition as someone suggested on the user list),
or introduce a DeleteOffsets API. Either way, this will probably need a KIP. 

> ConsumerGroupCommand should allow resetting offsets for consumer groups
> -----------------------------------------------------------------------
>                 Key: KAFKA-3059
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3059
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Gwen Shapira
>            Assignee: Jason Gustafson
> As discussed here:
> http://mail-archives.apache.org/mod_mbox/kafka-users/201601.mbox/%3CCA%2BndhHpf3ib%3Ddsh9zvtfVjRiUjSz%2B%3D8umXm4myW%2BpBsbTYATAQ%40mail.gmail.com%3E
> * Given a consumer group, remove all stored offsets
> * Given a group and a topic, remove offset for group  and topic
> * Given a group, topic, partition and offset - set the offset for the specified partition
and group with the given value

This message was sent by Atlassian JIRA

View raw message