kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Golovanov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-6537) Duplicate consumers after consumer group rebalance
Date Tue, 06 Feb 2018 12:39:00 GMT
Michael Golovanov created KAFKA-6537:

             Summary: Duplicate consumers after consumer group rebalance
                 Key: KAFKA-6537
                 URL: https://issues.apache.org/jira/browse/KAFKA-6537
             Project: Kafka
          Issue Type: Bug
          Components: consumer
    Affects Versions:
            Reporter: Michael Golovanov

*_Deployment description_*

Kafka brokers have been deployed on ten (10) nodes. Zookeeper cluster have seven (7) nodes.
Nodes of Kafka brokers shared with Zookeeper nodes on bare metal hosts.

Broker/Zookeeper hosts OS is Redhat 7 and JVM version is Java 8. Host names are grid48, grid237
и grid251.

We have one topic with six (6) patitions. Kafka consumers deployed on three (3) hosts. Each
host have two (2) consumers. All consumers belong to single group.


*_Error description_*

After start all consumers Apache Kafka partitions of topic was balanced evenly.

grid237 owns partitions 0,1 (0 - consumer thread-0, 1, consumer thread-1)
 grid251 owns partitions 2,3 (2 - consumer thread-0, consumer thread-1)
 grid48 owns partitions 4,5 (4- consumer thread-0, 5, consumer thread-1)

After some period of time we see haotic revokes and assigns partitions between brokers and
then all partitions in log assigned to one consumer on one node grid48


But really all partitions read not only by thread-1 consumer, but thread-0 on grid48. And
all messages from topic partitions was duplicate. Consumer thread-0 try to commit message
offset and get commit error, thread-1 consumer successfully commit offset.

This message was sent by Atlassian JIRA

View raw message