kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neha Narkhede (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1744) Fetch Response contains messages prior to the requested offset
Date Fri, 07 Nov 2014 01:41:35 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201394#comment-14201394

Neha Narkhede commented on KAFKA-1744:

The broker sends data to the consumer using zero-copy, so it cannot filter the extra messages
out. The spec already says Clients should handle this case. Should we close this JIRA?

> Fetch Response contains messages prior to the requested offset
> --------------------------------------------------------------
>                 Key: KAFKA-1744
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1744
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions:
>            Reporter: Evan Huus
> As reported in https://github.com/Shopify/sarama/issues/166 there are cases where a FetchRequest
for a particular offset returns some messages prior to that offset.
> The spec does not seem to indicate that this is possible; it does state that "As an optimization
the server is allowed to return a partial message at the end of the message set." but otherwise
implies that a request for offset X will only return complete messages starting at X. 
> The scala consumer does seem to handle this case gracefully though, if I am reading it
correctly (my scala is not the best): https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/consumer/ConsumerIterator.scala#L96-L99
> So is this a bug or just a case that needs to be added to the spec? Something like "As
an optimization the server is allowed to return some messages in the message set prior to
the requested offset. Clients should handle this case."? Although I can't imagine why sending
extra data would be faster than only sending the necessary messages...

This message was sent by Atlassian JIRA

View raw message