kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dana Powers (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-5465) FetchResponse v0 does not return any messages when max_bytes smaller than v2 message set
Date Sat, 17 Jun 2017 16:04:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dana Powers updated KAFKA-5465:
-------------------------------
    Description: 
In prior releases, when consuming uncompressed messages, FetchResponse v0 will returns a message
if it is smaller than the max_bytes sent in the FetchRequest. In 0.11.0.0 RC0, when messages
are stored as v2 internally, the response will be empty unless the full message set is smaller
than max_bytes. In some configurations, this may cause some old consumers to get stuck on
large messages where previously they were able to make progress one message at a time.

For example, when I produce 10 5KB messages using ProduceRequest v0 and then attempt FetchRequest
v0 with partition max bytes = 6KB (larger than a single message but smaller than all 10 messages
together), I get an empty message set from 0.11.0.0. Previous brokers would have returned
a single message.

  was:
In prior releases, when consuming uncompressed messages FetchResponse v0 will returns a message
if it is smaller than the max_bytes sent in the FetchRequest. In 0.11.0.0 RC0, when messages
are stored as v2 internally, the response will be empty unless the full message set is smaller
than max_bytes. In some configurations, this may cause some old consumers to get stuck on
large messages where previously they were able to make progress one message at a time.

For example, when I produce 10 5KB messages using ProduceRequest v0 and then attempt FetchRequest
v0 with partition max bytes = 6KB (larger than a single message but smaller than all 10 messages
together), I get an empty message set from 0.11.0.0. Previous brokers would have returned
a single message.


> FetchResponse v0 does not return any messages when max_bytes smaller than v2 message
set 
> -----------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5465
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5465
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.11.0.0
>            Reporter: Dana Powers
>            Priority: Minor
>
> In prior releases, when consuming uncompressed messages, FetchResponse v0 will returns
a message if it is smaller than the max_bytes sent in the FetchRequest. In 0.11.0.0 RC0, when
messages are stored as v2 internally, the response will be empty unless the full message set
is smaller than max_bytes. In some configurations, this may cause some old consumers to get
stuck on large messages where previously they were able to make progress one message at a
time.
> For example, when I produce 10 5KB messages using ProduceRequest v0 and then attempt
FetchRequest v0 with partition max bytes = 6KB (larger than a single message but smaller than
all 10 messages together), I get an empty message set from 0.11.0.0. Previous brokers would
have returned a single message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message