kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-3177) Kafka consumer can hang when position() is called on a non-existing partition.
Date Sat, 23 Sep 2017 04:49:06 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Guozhang Wang updated KAFKA-3177:

*Reminder to the contributor / reviewer of the PR*: please note that the code deadline for
1.0.0 is less than 2 weeks away (Oct. 4th). Please re-evaluate your JIRA and see if it still
makes sense to be merged into 1.0.0 or it could be pushed out to 1.1.0, or be closed directly
if the JIRA itself is not valid any more, or re-assign yourself as contributor / committer
if you are no longer working on the JIRA.

> Kafka consumer can hang when position() is called on a non-existing partition.
> ------------------------------------------------------------------------------
>                 Key: KAFKA-3177
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3177
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions:
>            Reporter: Jiangjie Qin
>            Assignee: Jason Gustafson
>            Priority: Critical
>             Fix For: 1.0.0
> This can be easily reproduced as following:
> {code}
> {
>     ...
>     consumer.assign(SomeNonExsitingTopicParition);
>     consumer.position();
>     ...
> }
> {code}
> It seems when position is called we will try to do the following:
> 1. Fetch committed offsets.
> 2. If there is no committed offsets, try to reset offset using reset strategy. in sendListOffsetRequest(),
if the consumer does not know the TopicPartition, it will refresh its metadata and retry.
In this case, because the partition does not exist, we fall in to the infinite loop of refreshing
topic metadata.
> Another orthogonal issue is that if the topic in the above code piece does not exist,
position() call will actually create the topic due to the fact that currently topic metadata
request could automatically create the topic. This is a known separate issue.

This message was sent by Atlassian JIRA

View raw message