kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4905) StreamPartitionAssignor doesn't respect subscriptions to assign partitions.
Date Thu, 31 Aug 2017 16:05:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149176#comment-16149176

Guozhang Wang commented on KAFKA-4905:

We can close it for now.

A general issue though, is that today we cannot provide a smooth upgrade path for applications
who has changed their topology subscribed topics online, due to this issue. If we want to
improve on this scenario we may need to revisit this later.

> StreamPartitionAssignor doesn't respect subscriptions to assign partitions.
> ---------------------------------------------------------------------------
>                 Key: KAFKA-4905
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4905
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions:
>            Reporter: Florian Hussonnois
> Both RangeAssignor and RoundRobinAssignor use the subscriptions to assign partition to
each consumer. This allow to have two consumers belonging to the the same group and subscribing
to two differents topics.
> This doesn't seem to be the case of the StreamPartitionAssignor resulting to an IllegalArgumentException
thrown during rebalance. 
> java.lang.IllegalArgumentException: Assigned partition foo-2 for non-subscribed topic
regex pattern; subscription pattern is bar
> 	at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignFromSubscribed(SubscriptionState.java:190)
> 	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:216)
> 	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:352)
> 	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:303)
> This is because the consumer group leader attempt to assign partitions to a consumer
that didn't subscribe to the associated topic.

This message was sent by Atlassian JIRA

View raw message