kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <...@confluent.io>
Subject Re: [DISCUSS] KIP-74: Add Fetch Response Size Limit in Bytes
Date Tue, 09 Aug 2016 17:46:22 GMT
Hi, Andrey,

Thanks for the proposal. It looks good overall. Some minor comments.

1. It seems that it's bit weird that fetch.partition.max.bytes is a broker
level configuration while fetch.limit.bytes is a client side configuration.
Intuitively, it seems both should be set by the client? If we do that, one
benefit is that we can validate that fetch.limit.bytes >=
fetch.partition.max.bytes on the client side.

2. Naming wise. fetch.response.max.bytes and replica.fetch.response.max.bytes
seem to be more consistent with our current convention than
fetch.limit.bytes and replica.fetch.limit.bytes.

3. When you say "This way we can ensure that response size is less than (
*limit_bytes* + *message.max.bytes*).", it should be "less than
max(limit_bytes, message.max.bytes)", right?

Finally, KIP-73 (replication quota) is proposing a similar change to fetch
request protocol. We can probably just combine the two changes into one,
instead of bumping the fetch request version twice.



On Mon, Aug 8, 2016 at 10:11 AM, Andrey L. Neporada <
aneporada@yandex-team.ru> wrote:

> Hi all!
> I’ve just created KIP-74: Add Fetch Response Size Limit in Bytes.
> The idea is to limit client memory consumption when fetching many
> partitions (especially useful for replication).
> Full details are here:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-74%3A+
> Add+Fetch+Response+Size+Limit+in+Bytes
> Thanks
> Andrey.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message