kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: Payload size exception
Date Tue, 29 Jan 2013 18:44:11 GMT
Ack, right you are Neha, my bad.

WRT to how to set the maximum there are two considerations:
1. It should be smaller then the fetch size your consumers use
2. Messages are fully instantiated in memory so obscenely large messages
(say hundreds of mb) will cause a lot of memory allocation churn/problems.

-Jay


On Tue, Jan 29, 2013 at 8:57 AM, S Ahmed <sahmed1020@gmail.com> wrote:

> Neha/Jay,
>
> At linkedin, what is the largest payload size per message you guys have in
> production?  My app might have like 20-100 kilobytes in size and I am
> hoping to get an idea if others have large messages like this for any
> production use case.
>
>
> On Tue, Jan 29, 2013 at 11:35 AM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > > In 0.7.x this
> > > setting is controlled by the broker configuration max.message.size.
> > >
> >
> > Actually, in 0.7.x this setting is controlled by max.message.size on the
> > producer. In 0.8, we moved this setting to the broker.
> >
> > Thanks,
> > Neha
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message