kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ismael Juma (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-4040) Round Robin Assignment does not create balanced assignments
Date Sat, 28 Oct 2017 15:55:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ismael Juma updated KAFKA-4040:
-------------------------------
    Affects Version/s:     (was: 0.8.2.2)
                       1.0.0
                       0.10.0.0
                       0.11.0.0

> Round Robin Assignment does not create balanced assignments
> -----------------------------------------------------------
>
>                 Key: KAFKA-4040
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4040
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions: 0.10.0.0, 0.11.0.0, 1.0.0
>            Reporter: Mathias Herberts
>
> The RoundRobinAssignor loops over the consumers in a circular manner, assigning each
time N partitions to each consumer, where N is the number of consuming threads.
> In the following scenario, this creates a major imbalance in the assignment.
> single topic with 64 partitions, 4 consumers (A,B,C,D), each with 12 threads. The roundrobin
strategy will allocate the partitions in the following manner:
> A: 24
> B: 16
> C: 12
> D: 12
> when the expected assignment would rather be 16 to each consumer.
> The reason for this imbalance is that instead of allocating a single partition to each
consumer in sequence, the assignor attempts to assign 12 partitions each time it is handed
a consumer by the circularIterator, so it starts by assigning 12 to each of A,B,C and D, then
it has 16 partitions left and it is handed A again, to which it assigns 12 partitions before
moving to B to which 4 will be assigned.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message