kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manikumar (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-1006) Consumer loses messages of a new topic with auto.offset.reset = largest
Date Mon, 30 Oct 2017 12:28:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Manikumar resolved KAFKA-1006.
------------------------------
    Resolution: Auto Closed

Closing inactive issue. The old consumer is no longer supported, please upgrade to the Java
consumer whenever possible.

> Consumer loses messages of a new topic with auto.offset.reset = largest
> -----------------------------------------------------------------------
>
>                 Key: KAFKA-1006
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1006
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8.0
>            Reporter: Swapnil Ghike
>            Assignee: Guozhang Wang
>              Labels: usability
>
> Consumer currently uses auto.offset.reset = largest by default. If a new topic is created,
consumer's topic watcher is fired. The consumer will first finish partition reassignment as
part of rebalance and then start consuming from the tail of each partition. Until the partition
reassignment is over, the server may have appended new messages to the new topic, consumer
won't consume these messages. Thus, multiple batches of messages may be lost when a topic
is newly created. 
> The fix is to start consuming from the earliest offset for newly created topics.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message