kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manikumar (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-275) max.message.size is not enforced for compressed messages
Date Thu, 17 Aug 2017 12:03:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Manikumar resolved KAFKA-275.
    Resolution: Fixed

This issue is fixed in latest versions.  Please reopen if the issue still exists. 

> max.message.size is not enforced for compressed messages
> --------------------------------------------------------
>                 Key: KAFKA-275
>                 URL: https://issues.apache.org/jira/browse/KAFKA-275
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.7
>            Reporter: Neha Narkhede
> The max.message.size check is not performed for compressed messages, but only for each
message that forms a compressed message. Due to this, even if the max.message.size is set
to 1MB, the producer can technically send n 1MB messages as one compressed message. This can
cause memory issues on the server as well as deserialization issues on the consumer. The consumer's
fetch size has to be > max.message.size in order to be able to read data. If one message
is larger than the fetch.size, the consumer will throw an exception and cannot proceed until
the fetch.size is increased. 
> Due to this bug, even if the fetch.size > max.message.size, the consumer can still
get stuck on a message that is larger than max.message.size.

This message was sent by Atlassian JIRA

View raw message