kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6723) Separate "max.poll.record" for restore consumer and common consumer
Date Wed, 28 Mar 2018 04:32:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416806#comment-16416806

Matthias J. Sax commented on KAFKA-6723:

Make sense to me. Seems to be related to KAFKA-6657 -- do you think KAFKA-6657 subsumes this

One open question, regardless of KAFKA-6657 is, if the default values for `max.poll.records`
should be different for both consumers.

> Separate "max.poll.record" for restore consumer and common consumer
> -------------------------------------------------------------------
>                 Key: KAFKA-6723
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6723
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Boyang Chen
>            Assignee: Boyang Chen
>            Priority: Minor
> Currently, Kafka Streams use `max.poll.record` config for both restore consumer and normal
stream consumer. In reality, they are doing different processing workloads, and in order to
speed up the restore speed, restore consumer is supposed to have a higher throughput by setting
`max.poll.record` higher. The change involved is trivial: [https://github.com/abbccdda/kafka/commit/cace25b74f31c8da79e93b514bcf1ed3ea9a7149]
> However, this is still a public API change (introducing a new config name), so we need
a KIP.

This message was sent by Atlassian JIRA

View raw message