kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srinivas Dhruvakumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4430) Broker logging "Topic and partition to exceptions: [topic,6] -> kafka.common.MessageSizeTooLargeException"
Date Tue, 22 Nov 2016 06:47:58 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15685885#comment-15685885

Srinivas Dhruvakumar commented on KAFKA-4430:

Sorry I forgot to mention. I have set the message.max.bytes on the Aggregate Kafka Cluster
to 1 MB. I could narrow it down to ProducerResponseCallback() in KafkaApi.scala where it is
logged as INFO. So was wondering if this is an actual error and if it is dropping messages.

> Broker logging "Topic and partition to exceptions: [topic,6] -> kafka.common.MessageSizeTooLargeException"
> ----------------------------------------------------------------------------------------------------------
>                 Key: KAFKA-4430
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4430
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions:
>         Environment: Production 
>            Reporter: Srinivas Dhruvakumar
>              Labels: newbie
> I have a setup as below 
> DC Kafka 
> Mirrormaker 
> Aggregate Kafka
> Here is the following settings. I have set the max.message.bytes to 1M Bytes on DC and
AGG kafka side. Mirrormaker producer settings --  batch.size is set to 500 K Bytes and max.request.size
is set to 1 M Byte and ack to 0 , compression-> gzip . 
> However on the Aggregate Kafka I get the following exception 
> Closing connection due to error during produce request with correlation id 414156659
from client id producer-1 with ack=0
> Topic and partition to exceptions: [topic1,6] -> kafka.common.MessageSizeTooLargeException
> Is this a bug or why would this happen. I have configured mirrormaker to send messages
less than 1 M Bytes . Are the messages getting dropped ? Under what circumstances this error
occurs ?

This message was sent by Atlassian JIRA

View raw message