kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vahid Hashemian (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (KAFKA-6434) Kafka-consumer-groups.sh reset-offsets does not work properly for not existing group
Date Sat, 13 Jan 2018 05:19:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324830#comment-16324830
] 

Vahid Hashemian edited comment on KAFKA-6434 at 1/13/18 5:18 AM:
-----------------------------------------------------------------

[~cabot] This is fixed in the current trunk (by KIP-171) and will land on 1.1.0.

It appears though that {{kafka-consumer-groups.sh --list --bootstrap-server localhost:9092}}
still does not output such groups because the protocol type for them is not set (the required
protocol type by this command is {{consumer}}). [~hachikuji], do you think this is the correct
behavior?

*Update*: I just came across [KAFKA-6287|https://issues.apache.org/jira/browse/KAFKA-6287]
which addresses the issue I mentioned.


was (Author: vahid):
[~cabot] This is fixed in the current trunk (by KIP-171) and will land on 1.1.0.

It appears though that {{kafka-consumer-groups.sh --list --bootstrap-server localhost:9092}}
still does not output such groups because the protocol type for them is not set (the required
protocol type by this command is {{consumer}}). [~hachikuji], do you think this is the correct
behavior?

> Kafka-consumer-groups.sh reset-offsets does not work properly for not existing group
> ------------------------------------------------------------------------------------
>
>                 Key: KAFKA-6434
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6434
>             Project: Kafka
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 0.11.0.2
>            Reporter: George Smith
>            Assignee: Vahid Hashemian
>
> Our usecase: We are migrating Spark streaming app into Kafka streaming. We want to continue
processing from the last processed offsets of the Spark streaming app. Therefore we want to
define new consumer group (application id) with given offsets. The new app was not launched
yet (we don't want to make side effects of processing into db) -> new consumer group does
not exist.
> I was happy to see the updated  Kafka-consumer-groups.sh supports reset-offsets method.
Unfortunately it seems it's not working as expected. 
> {code}
> kafka-consumer-groups.sh --reset-offsets --bootstrap-server localhost:9092 --topic testTopic:0
--group testGROUP --to-offset 10
> Note: This will only show information about consumers that use the Java consumer API
(non-ZooKeeper-based consumers).
> TOPIC                          PARTITION  NEW-OFFSET
> testTopic                      0          10
> {code}
> Now I want to check offsets for the group:
> {code}
> kafka-consumer-groups.sh --describe --bootstrap-server localhost:9092 --group testGROUP
> Error: Consumer group 'testGROUP' does not exist.
> {code}
> That's strange, isn't it?
> On the other side when I use kafka-streams-application-reset.sh - the group is obviously
created - unfortunately this tool does not support given offsets for partitions (only the
beginning is supported) + it does not support secured Kafka connection...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message