From jira-return-11282-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Wed Mar 28 20:06:03 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4DE87180652 for ; Wed, 28 Mar 2018 20:06:03 +0200 (CEST) Received: (qmail 15553 invoked by uid 500); 28 Mar 2018 18:06:02 -0000 Mailing-List: contact jira-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@kafka.apache.org Delivered-To: mailing list jira@kafka.apache.org Received: (qmail 15540 invoked by uid 99); 28 Mar 2018 18:06:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Mar 2018 18:06:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id F3A84C6F9F for ; Wed, 28 Mar 2018 18:06:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id WARz1v4MNwKe for ; Wed, 28 Mar 2018 18:06:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 36C4B5F5B3 for ; Wed, 28 Mar 2018 18:06:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id BED41E08C9 for ; Wed, 28 Mar 2018 18:06:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 3685F255E4 for ; Wed, 28 Mar 2018 18:06:00 +0000 (UTC) Date: Wed, 28 Mar 2018 18:06:00 +0000 (UTC) From: "Guozhang Wang (JIRA)" To: jira@kafka.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (KAFKA-6723) Separate "max.poll.record" for restore consumer and common consumer MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/KAFKA-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417856#comment-16417856 ] Guozhang Wang commented on KAFKA-6723: -------------------------------------- The proposal is to have a prefix for restore consumer and global consumer separate, not for a specific config only. I.e. for any consumer configs, users can prefix it to differentiate between different consumers, like: consumer.poll.ms restore-consumer.poll.ms global-consumer.poll.ms > Separate "max.poll.record" for restore consumer and common consumer > ------------------------------------------------------------------- > > Key: KAFKA-6723 > URL: https://issues.apache.org/jira/browse/KAFKA-6723 > Project: Kafka > Issue Type: Improvement > Components: streams > Reporter: Boyang Chen > Assignee: Boyang Chen > Priority: Minor > > Currently, Kafka Streams use `max.poll.record` config for both restore consumer and normal stream consumer. In reality, they are doing different processing workloads, and in order to speed up the restore speed, restore consumer is supposed to have a higher throughput by setting `max.poll.record` higher. The change involved is trivial: [https://github.com/abbccdda/kafka/commit/cace25b74f31c8da79e93b514bcf1ed3ea9a7149] > However, this is still a public API change (introducing a new config name), so we need a KIP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)