kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vahid Hashemian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4246) Discretionary partition assignment on the consumer side not functional
Date Mon, 03 Oct 2016 17:41:20 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542950#comment-15542950

Vahid Hashemian commented on KAFKA-4246:

[~nucatus] Could you share part of your code related to this issue?

> Discretionary partition assignment on the consumer side not functional
> ----------------------------------------------------------------------
>                 Key: KAFKA-4246
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4246
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions:
>            Reporter: Alexandru Ionita
> Trying to manually assign partition/topics to a consumer will not work correctly. The
consumer will be able to fetch records from the given partitions, but the first commit will
fail with the following message:
> {code}
> 2016-10-03 13:44:50.673 DEBUG 11757 --- [pool-9-thread-1] o.a.k.c.c.internals.ConsumerCoordinator
 : Offset commit for group XXXXXX failed: The coordinator is not aware of this member.
> 2016-10-03 13:44:50.673  WARN 11757 --- [pool-9-thread-1] o.a.k.c.c.internals.ConsumerCoordinator
 : Auto offset commit failed for group XXXXXX: Commit cannot be completed since the group
has already rebalanced and assigned the partitions to another member. This means that the
time between subsequent calls to poll() was longer than the configured session.timeout.ms,
which typically implies that the poll loop is spending too much time message processing. You
can address this either by increasing the session timeout or by reducing the maximum size
of batches returned in poll() with max.poll.records.
> {code}.
> All this while the consumer will continue to poll records from the kafka cluster, but
every commit will fail with the same message.
> I tried setting the {{session.timeout.ms}} to values like 50000, but I was getting the
same outcome => no successfull commits.
> If I only switch from {{consumer.assign( subscribedPartitions )}} to {{consumer.subscribe(
topics )}}, everything works as expected. No other client configurations should be changed
to make it work.
> Am I missing something here?

This message was sent by Atlassian JIRA

View raw message