kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apurva Mehta (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-5865) Expiring batches with idempotence enabled could cause data loss.
Date Tue, 26 Sep 2017 00:44:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apurva Mehta resolved KAFKA-5865.
---------------------------------
    Resolution: Fixed

This is fixed in 1.0.0 by the changes in https://github.com/apache/kafka/pull/3743

> Expiring batches with idempotence enabled could cause data loss.
> ----------------------------------------------------------------
>
>                 Key: KAFKA-5865
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5865
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.11.0.0
>            Reporter: Apurva Mehta
>            Assignee: Apurva Mehta
>             Fix For: 1.0.0
>
>
> Currently we have a problem with this line:
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java#L282
> Because we can reset the producer id and return after draining batches, it means that
we can drain batches for some partitions, then find a batch has expired, and then return.
But the batches which were drained are now no longer in the producer queue, and haven't been
sent. Thus they are totally lost, and the call backs will never be invoked.
> This is already fixed in https://github.com/apache/kafka/pull/3743 , but opening this
in case we want to fix it in 0.11.0.2 as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message