kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ismael Juma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-2478) KafkaConsumer javadoc example seems wrong
Date Thu, 27 Aug 2015 10:19:46 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716428#comment-14716428

Ismael Juma commented on KAFKA-2478:

[~devstr], yes, you can submit a PR. Instructions can be found here:


> KafkaConsumer javadoc example seems wrong
> -----------------------------------------
>                 Key: KAFKA-2478
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2478
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer
>    Affects Versions: 0.8.3
>            Reporter: Dmitry Stratiychuk
>            Assignee: Neha Narkhede
> I was looking at this KafkaConsumer example in the javadoc:
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L199
> As I understand, commit() method commits the maximum offsets returned by the most recent
invocation of poll() method.
> In this example, there's a danger of losing the data.
> Imagine the case where 300 records are returned by consumer.poll()
> The commit will happen after inserting 200 records into the database.
> But it will also commit the offsets for 100 records that are still unprocessed.
> So if consumer fails before buffer is dumped into the database again,
> then those 100 records will never be processed.
> If I'm wrong, could you please clarify the behaviour of commit() method?

This message was sent by Atlassian JIRA

View raw message