From dev-return-105303-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Tue Jun 25 18:55:12 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 5FC5218062F for ; Tue, 25 Jun 2019 20:55:12 +0200 (CEST) Received: (qmail 41196 invoked by uid 500); 25 Jun 2019 18:55:10 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 41185 invoked by uid 99); 25 Jun 2019 18:55:09 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Jun 2019 18:55:09 +0000 Received: from auth1-smtp.messagingengine.com (auth1-smtp.messagingengine.com [66.111.4.227]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id 7D5171C6F for ; Tue, 25 Jun 2019 18:55:09 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailauth.nyi.internal (Postfix) with ESMTP id 3EC17226CC for ; Tue, 25 Jun 2019 14:55:09 -0400 (EDT) Received: from imap1 ([10.202.2.51]) by compute2.internal (MEProxy); Tue, 25 Jun 2019 14:55:09 -0400 X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeggddufeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpedfveholhhinhcuofgtvegrsggvfdcuoegtmhgttggrsggv segrphgrtghhvgdrohhrgheqnecuffhomhgrihhnpegrphgrtghhvgdrohhrghenucfrrg hrrghmpehmrghilhhfrhhomheptghmtggtrggsvgefudegodhmvghsmhhtphgruhhthhhp vghrshhonhgrlhhithihqdegiedukeegudeftddqudehheekkeehudegqdgtmhgttggrsg gvpeeprghprggthhgvrdhorhhgsehfrghsthhmrghilhdrtghomhenucevlhhushhtvghr ufhiiigvpedt X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 0636DC200A4; Tue, 25 Jun 2019 14:55:09 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.1.6-730-g63f2c3b-fmstable-20190622v1 Mime-Version: 1.0 Message-Id: <59af0fd7-33f6-42b1-8b25-d680379ef62d@www.fastmail.com> In-Reply-To: References: <56a4ad42-4336-419f-bd3d-ba0b40f68c06@www.fastmail.com> Date: Tue, 25 Jun 2019 11:54:37 -0700 From: "Colin McCabe" To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-480 : Sticky Partitioner Content-Type: text/plain Hi Justine, The KIP discusses adding a new method to the partitioner interface. > default public Integer onNewBatch(String topic, Cluster cluster) { ... } However, this new method doesn't give the partitioner access to the key and value of the message. While this works for the case described here (no key), in general we might need this information when re-assigning a partitition based on the batch completing. So I think we should add these methods to onNewBatch. Also, it would be nice to call this something like "repartitionOnNewBatch" or something, to make it clearer what is going on. best, Colin On Mon, Jun 24, 2019, at 18:32, Boyang Chen wrote: > Thank you Justine for the KIP! Do you mind creating a corresponding JIRA > ticket too? > > On Mon, Jun 24, 2019 at 4:51 PM Colin McCabe wrote: > > > Hi Justine, > > > > Thanks for the KIP. This looks great! > > > > In one place in the KIP, you write: "Remove > > testRoundRobinWithUnavailablePartitions() and testRoundRobin() since the > > round robin functionality of the partitioner has been removed." You can > > skip this and similar lines. We don't need to describe changes to internal > > test classes in the KIP since they're not visible to users or external > > developers. > > > > It seems like maybe the performance tests should get their own section. > > Right now, the way the layout is makes it look like they are part of the > > "Compatibility, Deprecation, and Migration Plan" > > > > best, > > Colin > > > > > > On Mon, Jun 24, 2019, at 14:04, Justine Olshan wrote: > > > Hello, > > > This is the discussion thread for KIP-480: Sticky Partitioner. > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner > > > > > > Thank you, > > > Justine Olshan > > > > > >