kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Evan Huus (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-1744) Fetch Response contains messages prior to the requested offset
Date Sat, 01 Nov 2014 12:48:33 GMT
Evan Huus created KAFKA-1744:
--------------------------------

             Summary: Fetch Response contains messages prior to the requested offset
                 Key: KAFKA-1744
                 URL: https://issues.apache.org/jira/browse/KAFKA-1744
             Project: Kafka
          Issue Type: Bug
          Components: core
    Affects Versions: 0.8.1.1
            Reporter: Evan Huus


As reported in https://github.com/Shopify/sarama/issues/166 there are cases where a FetchRequest
for a particular offset returns some messages prior to that offset.

The spec does not seem to indicate that this is possible; it does state that "As an optimization
the server is allowed to return a partial message at the end of the message set." but otherwise
implies that a request for offset X will only return complete messages starting at X. 

The scala consumer does seem to handle this case gracefully though, if I am reading it correctly
(my scala is not the best): https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/consumer/ConsumerIterator.scala#L96-L99

So is this a bug or just a case that needs to be added to the spec? Something like "As an
optimization the server is allowed to return some messages in the message set prior to the
requested offset. Clients should handle this case."? Although I can't imagine why sending
extra data would be faster than only sending the necessary messages...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message