kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Partial message when packet size = max size?
Date Wed, 04 Jan 2012 01:37:50 GMT
Aaron,

The consumer has to use a fetch size larger than the largest single message
(+ overhead, about 10 bytes). Otherwise, the consumer won't be able to
decode a single message from a fetched chunk and will get stuck.

Thanks,

Jun

On Tue, Jan 3, 2012 at 2:08 PM, Aaron Raddon <araddon@gmail.com> wrote:

> Hello;
>
> I am using the golang Consumer and ran into some errors when there was
> a large backlog (greater than max message size) of messages on broker
> and had a question about the format.
>
> In the case above, the behavior i was seeing was the total packet
> bytes size == max message (containing many messages), even if this
> meant a partial/incomplete last message.   Is this correct?
>
> I wrote a fix for it in the driver, but had second thoughts if this was
> right.
>
> My Kafka broker was built from source ~ 1 week ago.
>
> Thanks
>
> Aaron
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message