kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Fell (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-6554) Broker doesn't reject Produce request with inconsistent state
Date Mon, 12 Feb 2018 19:30:00 GMT
Simon Fell created KAFKA-6554:
---------------------------------

             Summary: Broker doesn't reject Produce request with inconsistent state
                 Key: KAFKA-6554
                 URL: https://issues.apache.org/jira/browse/KAFKA-6554
             Project: Kafka
          Issue Type: Bug
          Components: producer 
    Affects Versions: 1.0.0
            Reporter: Simon Fell
         Attachments: produce_v3.txt

Produce messages of type v3 have offset deltas in each record along with a LastOffsetDelta
for the topic/partition set. In investigating an issue with missing offsets, I found a bug
in a producer library where it would send multiple records, but leave LastOffsetDelta at 0.
This causes various problems including holes in the offsets fetched by the consumer. 

As lastOffsetDelta can be computed by looking at the records, it seems like the broker should
at least validate the LastOffsetDelta field against the contained records to stop this bad
data getting in.

I've attached a decode v3 produce message that was causing the problems, and was accepted
by the broker.

Here's a link to the issue in the kafka library we were using which has more context if you
need it.

https://github.com/Shopify/sarama/issues/1032

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message