kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6723) Separate "max.poll.record" for restore consumer and common consumer
Date Wed, 28 Mar 2018 18:06:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417856#comment-16417856
] 

Guozhang Wang commented on KAFKA-6723:
--------------------------------------

The proposal is to have a prefix for restore consumer and global consumer separate, not for
a specific config only. I.e. for any consumer configs, users can prefix it to differentiate
between different consumers, like:

consumer.poll.ms
restore-consumer.poll.ms
global-consumer.poll.ms

> Separate "max.poll.record" for restore consumer and common consumer
> -------------------------------------------------------------------
>
>                 Key: KAFKA-6723
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6723
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Boyang Chen
>            Assignee: Boyang Chen
>            Priority: Minor
>
> Currently, Kafka Streams use `max.poll.record` config for both restore consumer and normal
stream consumer. In reality, they are doing different processing workloads, and in order to
speed up the restore speed, restore consumer is supposed to have a higher throughput by setting
`max.poll.record` higher. The change involved is trivial: [https://github.com/abbccdda/kafka/commit/cace25b74f31c8da79e93b514bcf1ed3ea9a7149]
> However, this is still a public API change (introducing a new config name), so we need
a KIP.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message