kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Boyang Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6723) Separate "max.poll.record" for restore consumer and common consumer
Date Wed, 28 Mar 2018 16:33:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417697#comment-16417697
] 

Boyang Chen commented on KAFKA-6723:
------------------------------------

[~mjsax]  [~guozhang] I haven't seen a concrete design for KAFKA-6657. So am I expecting
sth like:

default.max.poll.records

restore.max.poll.records

global.max.poll.records

If that one could cover cases by creating new consumer config parameters, I guess this Jira
sounds duplicate. Btw, I think the default value should be the same.

 

> Separate "max.poll.record" for restore consumer and common consumer
> -------------------------------------------------------------------
>
>                 Key: KAFKA-6723
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6723
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Boyang Chen
>            Assignee: Boyang Chen
>            Priority: Minor
>
> Currently, Kafka Streams use `max.poll.record` config for both restore consumer and normal
stream consumer. In reality, they are doing different processing workloads, and in order to
speed up the restore speed, restore consumer is supposed to have a higher throughput by setting
`max.poll.record` higher. The change involved is trivial: [https://github.com/abbccdda/kafka/commit/cace25b74f31c8da79e93b514bcf1ed3ea9a7149]
> However, this is still a public API change (introducing a new config name), so we need
a KIP.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message