kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Cheng (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-7144) Kafka Streams doesn't properly balance partition assignment
Date Tue, 10 Jul 2018 05:13:00 GMT
James Cheng created KAFKA-7144:

             Summary: Kafka Streams doesn't properly balance partition assignment
                 Key: KAFKA-7144
                 URL: https://issues.apache.org/jira/browse/KAFKA-7144
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 1.1.0
            Reporter: James Cheng
         Attachments: OneThenTwelve.java

Kafka Streams doesn't always spread the tasks across all available instances/threads

I have a topology which consumes a single partition topic and goes .through() a 12 partition
topic. The makes 13 partitions.


I then started 2 instances of the application. I would have expected the 13 partitions to
be split across the 2 instances roughly evenly (7 partitions on one, 6 partitions on the other).

Instead, one instance gets 12 partitions, and the other instance gets 1 partition.


Repro case attached. I ran it a couple times, and it was fairly repeatable.

Setup for the repro:
$ ./bin/kafka-topics.sh --zookeeper localhost --create --topic one --partitions 1 --replication-factor
$ ./bin/kafka-topics.sh --zookeeper localhost --create --topic twelve --partitions 12 --replication-factor
$ echo foo | kafkacat -P -b -t one

This message was sent by Atlassian JIRA

View raw message