From dev-return-101455-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Tue Jan 29 21:00:20 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 7127118067E for ; Tue, 29 Jan 2019 22:00:19 +0100 (CET) Received: (qmail 92005 invoked by uid 500); 29 Jan 2019 21:00:18 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 91993 invoked by uid 99); 29 Jan 2019 21:00:18 -0000 Received: from mail-relay.apache.org (HELO mailrelay2-lw-us.apache.org) (207.244.88.137) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Jan 2019 21:00:18 +0000 Received: from auth2-smtp.messagingengine.com (auth2-smtp.messagingengine.com [66.111.4.228]) by mailrelay2-lw-us.apache.org (ASF Mail Server at mailrelay2-lw-us.apache.org) with ESMTPSA id B2A5D2DD4 for ; Tue, 29 Jan 2019 21:00:17 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailauth.nyi.internal (Postfix) with ESMTP id 61F9E235AE for ; Tue, 29 Jan 2019 16:00:17 -0500 (EST) Received: from imap1 ([10.202.2.51]) by compute2.internal (MEProxy); Tue, 29 Jan 2019 16:00:17 -0500 X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtledrjedvgddugeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfquhhtnecuuegrihhlohhuthemucef tddtnecunecujfgurhepofgfkfgjfhffhffvufgtgfesthhqredtreerjeenucfhrhhomh epfdevohhlihhnucfotgevrggsvgdfuceotghmtggtrggsvgesrghprggthhgvrdhorhhg qeenucfrrghrrghmpehmrghilhhfrhhomheptghmtggtrggsvgefudegodhmvghsmhhtph gruhhthhhpvghrshhonhgrlhhithihqdegiedukeegudeftddqudehheekkeehudegqdgt mhgttggrsggvpeeprghprggthhgvrdhorhhgsehfrghsthhmrghilhdrtghomhenucevlh hushhtvghrufhiiigvpedt X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 083E7D467F; Tue, 29 Jan 2019 16:00:17 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.1.5-766-ge2be361-fmstable-20190123v1 X-Me-Personality: 46184130 Message-Id: In-Reply-To: References: <1545468796.3434321.1616090552.1E140647@webmail.messagingengine.com> Date: Tue, 29 Jan 2019 16:00:10 -0500 From: "Colin McCabe" To: dev@kafka.apache.org Subject: =?UTF-8?Q?Re:_[EXTERNAL]_-_Re:_[DISCUSS]_KIP-387:_Fair_Message_Consumpti?= =?UTF-8?Q?on_Across_Partitions_in_KafkaConsumer?= Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable On Mon, Jan 28, 2019, at 06:57, ChienHsing Wu wrote: > So... Does this non-response mean I should drop this topic after almos= t=20 > one month, folks? Hi ChienHsing, I would recommend dropping it, since I don't see a lot of uses for it. = Maybe there is something I missed, though. See my responses below: >=20 > -----Original Message----- > From: ChienHsing Wu =20 > Sent: Monday, January 21, 2019 12:47 PM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20 > Consumption Across Partitions in KafkaConsumer >=20 > Hi all, not sure what to do next as weeks have gone by, guys. --CH >=20 > -----Original Message----- > From: ChienHsing Wu > Sent: Monday, January 14, 2019 9:13 AM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20 > Consumption Across Partitions in KafkaConsumer >=20 > Hi, >=20 > I know everyone is busy. But I would appreciate someone letting me kno= w=20 > what to do next. I started this effort back in last year early=20 > November... >=20 > Thanks, CH >=20 > -----Original Message----- > From: ChienHsing Wu > Sent: Monday, January 07, 2019 9:24 AM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20 > Consumption Across Partitions in KafkaConsumer >=20 > Hi guys, >=20 > I am not sure what to do next in this KIP process. Could anyone please= =20 > help/advise me on what to do next?=20 >=20 > Thanks, CH >=20 > -----Original Message----- > From: ChienHsing Wu > Sent: Wednesday, January 02, 2019 12:55 PM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20 > Consumption Across Partitions in KafkaConsumer >=20 > Hi Colin, >=20 > Setting max.partition.fetch.bytes was discussed in the ticket. It's no= t=20 > as desirable if the message size is highly variable. Also this decreas= e=20 > the efficiency of network communication.=20 Even if the message size is highly variable, you can still set the max.p= artition.fetch.bytes to a very small value., perhaps even 1. Since we a= lways return at least one record, regardless of size limits, this will h= ave the effect you want. I agree that this may be less efficient than f= etching a larger number of records. But it will get you true round-robi= n behavior. >=20 > In the case you mentioned below where a consumer can get messages from= =20 > A, B, C and D but the consumer currently only has messages from A, B=20= > and C, the proposed change will NOT wait until some messages from D=20= > arrives to start returning messages; it will just serve those from A, = B=20 > and It will include those from D when they are available. That IS=20 > the current behavior. The proposed change does not impose a strict=20 > round robin pattern. I guess this is debatable, but the behavior you're proposing doesn't see= m "fairer" than what we do now. Imagine you are subscribed to 100 parti= tions and each fetch request only gives you back records from 3 of them.= Then the order the consumer sees records might be something like: ABCABCABCDEFDEFDEFDEFGHIGHIGHI... etc. Is this fairer than AAABBBCCCDDDEEEFFFGGGHHHIII? Seems questionable. It feels like maybe what you really want is a work queue for A, B, C, D,= etc. so that messages from different partitions can be processed in par= allel. And then perhaps pause fetching a partition when the work queue = for that partition grows too long. >=20 > The original KIP 41 discussed "Ensuring Fair Consumption", that means=20= > it originally intended to take that into account in the Consumer code,= =20 > the proposed change takes the current algorithm closer to that goal,=20= > IMHO. I could implement that logic at the caller side but, that would=20= > mean each library user need to know the inner working of the consumer=20= > code and to implement the logic on their own. Though as a first timer=20= > here, I do appreciate the complexity and functionalities in the client= =20 > library and feel that we'd be better off as a community to implement=20= > the logic in the library so the complexity is hidden from library user= s. The discussion in KIP-41 in the "ensuring fair consumption" section is a= bout making sure that no partitions get starved forever. This would hap= pen if, for example, we just constantly fetched from a single partition = and never fetched from some other partition. The discussion in that KIP= isn't about interleaving the partition order of the buffered records we= return to the consumer. best, Colin >=20 > Thanks, CH >=20 > -----Original Message----- > From: Colin McCabe > Sent: Saturday, December 22, 2018 3:53 AM > To: dev@kafka.apache.org > Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20 > Consumption Across Partitions in KafkaConsumer >=20 > Hi ChienHsing Wu, >=20 > Maybe I'm misunderstanding something, but I'm not sure I see the need=20= > for a KIP here. You can just set max.partition.fetch.bytes to a very=20= > small value. That will cause Kafka to fetch only one message from eac= h=20 > partition. This will give you the round robin behavior you want. >=20 > Alternately, if you don't want to change max.partition.fetch.bytes, yo= u=20 > could do your own buffering to get round robin behavior. Keep a buffe= r=20 > of messages from partition A, B, C, and D and hold back the messages=20= > from A, B, and C until one from D arrives, so that the A B C D A B C=20= > D... etc. order always repeats. >=20 > best, > Colin >=20 >=20 > On Wed, Dec 19, 2018, at 09:00, ChienHsing Wu wrote: > > Looking back the email thread I think one of the comments from=20 > > Mayuresh was the question about needing KIP for this change or not a= s=20 > > the KafkaConsumer does not guarantee the end user any order, and so = no=20 > > changes to the contracts to users. > >=20 > > I entered KIP based on suggestions from the attached email when goin= g=20 > > through code contribution process. I am not sure what to do next in=20= > > this KIP process. Could anyone please help/advise me on what to do n= ext? > >=20 > > Thanks! > >=20 > > CH > >=20 > > -----Original Message----- > > From: ChienHsing Wu > > Sent: Wednesday, December 12, 2018 1:05 PM > > To: dev@kafka.apache.org > > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20 > > Consumption Across Partitions in KafkaConsumer > >=20 > > Good to know that, Thanks!=20 > >=20 > > Nonetheless, that introduces additional complexity at the client sid= e=20 > > for a common expectation to more or less receives records in a fair=20= > > fashion. > >=20 > > CH > >=20 > > -----Original Message----- > > From: Mayuresh Gharat > > Sent: Wednesday, December 12, 2018 12:55 PM > > To: dev@kafka.apache.org > > Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20 > > Consumption Across Partitions in KafkaConsumer > >=20 > > Hi ChienHsing, > >=20 > > We are actually working on buffering the already fetched data for=20= > > paused topicPartitions, so ideally it should not have any effect on=20= > > performance. > > Associated jira :=20 > > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__issues.apache= .org > > _jira_browse_KAFKA-2D7548&d=3DDwIFaQ&c=3DZgVRmm3mf2P1-XDAyDsu4A&r=3D= Az03wMrb > > L9ToLW0OFyo3wo3985rhAKPMLmmg6RA3V7I&m=3D7eC1W-f8nKkXMGJti3n0zF4qDV0a= f8y5 > > uOWVIftTJ-U&s=3D_ERDVQqqt9Grnxt7DDO_gC9CvpD_ylhH8ZoHLwSXEpU&e=3D > >=20 > > Thanks, > >=20 > > Mayuresh > >=20 > > On Wed, Dec 12, 2018 at 6:01 AM ChienHsing Wu wrote: > >=20 > > > Hi Mayuresh, > > > > > > Thanks for the input! > > > > > > Pausing and Resuming are cumbersome and has some undesirable=20 > > > performance impact since pausing will in effect clean up the=20 > > > completed fetch and resuming will call the broker to retrieve agai= n. > > > > > > The way I changed the code was just to parse the completed fetch=20= > > > earlier and ensure the order to retrieve are the same as the compl= eted fetch queue. > > > I did make code changes to take into account the following in Fetc= her class. > > > > > > 1) exception handling > > > 2) ensure the parsed partitions are not included in=20 > > > fetchablePartitions > > > 3) clear buffer when not in the newly assigned partitions in=20 > > > clearBufferedDataForUnassignedPartitions > > > 4) close them properly in close method > > > > > > Though the consumer does not guarantee explicit order, KIP 41 (lin= k > > > below) did intend to ensure fair distribution and therefore the=20= > > > round robin algorithm in the code. The change I propose was to enh= ance it. > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__cwiki.apach= e.or > > > g_ > > > confluence_display_KAFKA_KIP-2D41-253A-2BKafkaConsumer-2BMax-2BRec= or > > > ds > > > -23KIP-2D41-3AKafkaConsumerMaxRecords-2DEnsuringFairConsumption&d=3D= Dw > > > IF > > > aQ&c=3DZgVRmm3mf2P1-XDAyDsu4A&r=3DAz03wMrbL9ToLW0OFyo3wo3985rhAKPM= Lmmg6R > > > A3 > > > V7I&m=3D7eC1W-f8nKkXMGJti3n0zF4qDV0af8y5uOWVIftTJ-U&s=3DNKZHA5HVgg= fKWlF_ > > > yg > > > 6V3-Wyf_Z6x7n1HQPQ1_M0d9A&e=3D > > > > > > As for performance, the changes does not add any additional calls = to=20 > > > the broker nor does it introduce significant processing logic; it=20= > > > just parses the completed fetch earlier and have a list to manage = them. > > > > > > > > > CH > > > > > > -----Original Message----- > > > From: Mayuresh Gharat > > > Sent: Tuesday, December 11, 2018 6:58 PM > > > To: dev@kafka.apache.org > > > Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20 > > > Consumption Across Partitions in KafkaConsumer > > > > > > Hi ChienHsing, > > > > > > The other way I was thinking, this can be done outside of=20 > > > KafkaConsumer is by pausing and resuming TopicPartitions (may be i= n round robin fashion). > > > There is some gotcha there as in you might not know if the consume= r=20 > > > has already fetched data for the remaining partitions. > > > Also I am not sure, if we need a KIP for this as the KafkaConsumer= =20 > > > does not guarantee the end user, any order, I believe. So if this=20= > > > change goes in, I don't think its changing the underlying behavior= . > > > It would be good to check if this change will impact the performan= ce=20 > > > of the consumer. > > > > > > Thanks, > > > > > > Mayuresh > > > > > > > > > On Tue, Dec 11, 2018 at 11:03 AM ChienHsing Wu=20 > > > > > > wrote: > > > > > > > Hi Mayuresh, > > > > > > > > To serve one poll call the logic greedily gets records from one=20= > > > > completed fetch before including records from the next completed= =20 > > > > fetch from the queue, as you described. > > > > > > > > The algorithm remembers the current completed fetch as starting=20= > > > > one when serving the next poll call. The net effect is that=20 > > > > completed fetch will be retrieved to serve as many poll calls=20= > > > > before retrieving records from any other completed fetches. > > > > > > > > For example, let's say the consumer has been assigned partition = A,=20 > > > > B and C and the max.poll.records is set to 100. Right now we hav= e=20 > > > > completed fetch A, and B. Each one has 300 records. It will take= 6=20 > > > > poll calls to retrieve all record and the sequence of retrieved=20= > > > > partitions will be: A, A, A, B, B, B. > > > > > > > > Ideally, it should alternate between A and B. I was proposing to= =20 > > > > move to the next one fetch for the next poll call based on the=20= > > > > order in the completed fetch queue, so the order becomes A, B, A= , B, A, B. > > > > The implementation parses the completed fetch only once. > > > > > > > > Thanks, CH > > > > > > > > -----Original Message----- > > > > From: Mayuresh Gharat > > > > Sent: Tuesday, December 11, 2018 1:21 PM > > > > To: dev@kafka.apache.org > > > > Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20= > > > > Consumption Across Partitions in KafkaConsumer > > > > > > > > Hi ChienHsing, > > > > > > > > Thanks for the KIP. > > > > It would be great if you can explain with an example, what you m= ean by " > > > > Currently the implementation will return available records=20 > > > > starting from the last partition the last poll call retrieves re= cords from. > > > > This leads to unfair patterns of record consumption from multipl= e > > > partitions." > > > > > > > > KafkaConsumer would send fetch requests to multiple brokers and=20= > > > > then gets the corresponding responses and puts them in to a sing= le=20 > > > > queue of CompletedFetches. IT then iterates over these completed= =20 > > > > fetches queue and peels of number of records =3D max.poll.record= s=20 > > > > from each completedFetch for each poll() before moving on to nex= t=20 > > > > completedFetch. Also it does not send a fetch request for a=20 > > > > TopicPartition, if we already have a buffered data (completedFet= ch=20 > > > > or > > > > nextInlineRecord) for that TopicPartition. It also moves the=20 > > > > TopicPartition to the end of the assignment queue, once it has=20= > > > > received data from broker for that TopicPartition, to maintain=20= > > > > round > > > robin fetch sequence for fairness. > > > > > > > > Thanks, > > > > > > > > Mayuresh > > > > > > > > On Tue, Dec 11, 2018 at 9:13 AM ChienHsing Wu=20 > > > > > > > > wrote: > > > > > > > > > Jason, > > > > > > > > > > > > > > > > > > > > KIP 41 was initiated by you and this KIP is to change the logi= c=20 > > > > > discussed in the Ensure Fair Consumption< > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__cwiki.apa= che. > > > > or > > > > g_ > > > > confluence_display_KAFKA_KIP-2D41-253A-2BKafkaConsumer-2BMax-2BR= ec > > > > or > > > > ds > > > > -23KIP-2D41-3AKafkaConsumerMaxRecords-2DEnsuringFairConsumption&= d=3D > > > > Dw > > > > IF > > > > aQ&c=3DZgVRmm3mf2P1-XDAyDsu4A&r=3DAz03wMrbL9ToLW0OFyo3wo3985rhAK= PMLmmg > > > > 6R > > > > A3 > > > > V7I&m=3DjeijHrRehjaysSML7ZSVlVEepS5LWchozwVVbwp7TLA&s=3DwarXH2nt= tWvhdQ > > > > hn > > > > -o > > > > SZuBYfZ_V2OY5ikbksVMzbt9o&e=3D > > > > >. > > > > > Your input on KIP-387< > > > > > > > > > >https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__cwiki.ap= ache > > > > >.o > > > > >rg > > > > >_ > > > > >confluence_display_KAFKA_KIP-2D387-253A-2BFair-2BMessage-2BCons= um > > > > >pt > > > > >io > > > > >n > > > > >-2BAcross-2BPartitions-2Bin-2BKafkaConsumer&d=3DDwIFaQ&c=3DZgVR= mm3mf2 > > > > >P1 > > > > >-X > > > > >D > > > > >AyDsu4A&r=3DAz03wMrbL9ToLW0OFyo3wo3985rhAKPMLmmg6RA3V7I&m=3Djei= jHrReh > > > > >ja > > > > >ys > > > > >S > > > > >ML7ZSVlVEepS5LWchozwVVbwp7TLA&s=3DPtfb85HFvz0TqKSju21-_uV-U_0_H= HNln > > > > >Nf > > > > >0k > > > > >T > > > > > tRlgk&e=3D> > > > > > would be very valuable. > > > > > > > > > > > > > > > > > > > > Thanks, ChienHsing > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: ChienHsing Wu > > > > > Sent: Tuesday, December 04, 2018 11:43 AM > > > > > To: dev@kafka.apache.org > > > > > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20= > > > > > Consumption Across Partitions in KafkaConsumer > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > Any comments/updates? I am not sure the next steps if no one h= as=20 > > > > > any further comments. > > > > > > > > > > > > > > > > > > > > Thanks, CH > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: ChienHsing Wu > > > > > > > > > > > > > > > > Sent: Tuesday, November 20, 2018 2:46 PM > > > > > > > > > > To: dev@kafka.apache.org > > > > > > > > > > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20= > > > > > Consumption Across Partitions in KafkaConsumer > > > > > > > > > > > > > > > > > > > > Hi Matt, > > > > > > > > > > > > > > > > > > > > Thanks for the feedback. > > > > > > > > > > > > > > > > > > > > The issue with the current design is that it stays on the=20 > > > > > previous partition even if the last poll call consumes the=20 > > > > > max.poll.records; it will consume all records in that partitio= n=20 > > > > > available at the consumer side to serve multiple poll calls=20= > > > > > before moving to the next > > > partition. > > > > > > > > > > > > > > > > > > > > Introducing another threshold at partition level will decrease= =20 > > > > > the number of records consumed in one partition within one pol= l=20 > > > > > call but will still use that same partition as the starting on= e=20 > > > > > in the next poll > > > > call. > > > > > > > > > > > > > > > > > > > > The same effect can be achieved by setting max.poll.records to= > > > > > 100 I believe. The main difference is that the client will nee= d=20 > > > > > to make more poll calls when that value is set to 100, and=20 > > > > > because of the non-blocking nature I believe the cost of extra= =20 > > > > > poll calls are not > > > > significant. > > > > > > > > > > > > > > > > > > > > Further thoughts? > > > > > > > > > > > > > > > > > > > > Thanks, CH > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: Matt Farmer > > > > > > > > > > > Sent: Monday, November 19, 2018 9:32 PM > > > > > > > > > > To: dev@kafka.apache.org > > > > > > > > > > Subject: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message=20 > > > > > Consumption Across Partitions in KafkaConsumer > > > > > > > > > > > > > > > > > > > > Hi there, > > > > > > > > > > > > > > > > > > > > Thanks for the KIP. > > > > > > > > > > > > > > > > > > > > We=E2=80=99ve run into issues with this at Mailchimp so someth= ing to=20 > > > > > address consuming behavior would save us from having to always= =20 > > > > > ensure we=E2=80=99re running enough consumers that each consum= er has=20 > > > > > only one partition (which is our usual MO). > > > > > > > > > > > > > > > > > > > > I wonder though if it would be simpler and more powerful to=20= > > > > > define the maximum number of records the consumer should pull=20= > > > > > from one partition before pulling some records from another? > > > > > > > > > > > > > > > > > > > > So if you set max.poll.records to 500 and then some new settin= g,=20 > > > > > max.poll.records.per.partition, to 100 then the Consumer would= =20 > > > > > switch what partition it reads from every 100 records - loopin= g=20 > > > > > back around to the first partition that had records if there=20= > > > > > aren=E2=80=99t 5 or more partitions with records. > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > On Mon, Nov 19, 2018 at 9:11 AM ChienHsing Wu=20 > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi, could anyone please review this KIP? > > > > > > > > > > > > > > > > > > > > > > Thanks, ChienHsing > > > > > > > > > > > > > > > > > > > > > > From: ChienHsing Wu > > > > > > > > > > > Sent: Friday, November 09, 2018 1:10 PM > > > > > > > > > > > To: dev@kafka.apache.org > > > > > > > > > > > Subject: RE: [DISCUSS] KIP-387: Fair Message Consumption=20 > > > > > > Across > > > > > > > > > > > Partitions in KafkaConsumer > > > > > > > > > > > > > > > > > > > > > > Just to check: Will anyone review this? It's been silent for= a > > > week... > > > > > > > > > > > Thanks, ChienHsing > > > > > > > > > > > > > > > > > > > > > > From: ChienHsing Wu > > > > > > > > > > > Sent: Monday, November 05, 2018 4:18 PM > > > > > > > > > > > To: 'dev@kafka.apache.org' > > > > > > > > > > dev@kafka.apache.org>> > > > > > > > > > > > Subject: [DISCUSS] KIP-387: Fair Message Consumption Across=20= > > > > > > Partitions > > > > > > > > > > > in KafkaConsumer > > > > > > > > > > > > > > > > > > > > > > Hi I just put together the KIP page as requested. This email= =20 > > > > > > is to > > > > > > > > > > > start the discussion thread. > > > > > > > > > > > > > > > > > > > > > > KIP: KIP-387: Fair Message Consumption Across Partitions in > > > > > > > > > > > KafkaConsumer< > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__cwiki= .apache. > > > > > > or > > > > > > g_ > > > > > > > > > > > confluence_display_KAFKA_KIP-2D387-253A-2BFair-2BMessage-2BC= on > > > > > > su > > > > > > mp > > > > > > ti > > > > > > on > > > > > > > > > > > -2BAcross-2BPartitions-2Bin-2BKafkaConsumer&d=3DDwIFaQ&c=3DZ= gVRmm3 > > > > > > mf > > > > > > 2P > > > > > > 1- > > > > > > XD > > > > > > > > > > > AyDsu4A&r=3DAz03wMrbL9ToLW0OFyo3wo3985rhAKPMLmmg6RA3V7I&m=3D= HhHPjf > > > > > > cr > > > > > > k8 > > > > > > Xi > > > > > > oE > > > > > > > > > > > m16n75UIKYwi8c8YrzVrp5tBK7LX8&s=3DgBGG4GvzPu-xhQ-uUqlq30U-bz= wcKZ > > > > > > _l > > > > > > NP > > > > > > 1b > > > > > > F5 > > > > > > > > > > > 49_KU&e=3D > > > > > > > > > > > > > > > > > > > > > > > Pull Request: > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__githu= b.co > > > > > > m_ > > > > > > ap > > > > > > ac > > > > > > he > > > > > > > > > > > _kafka_pull_5838&d=3DDwIFaQ&c=3DZgVRmm3mf2P1-XDAyDsu4A&r=3DA= z03wMrbL > > > > > > 9T > > > > > > oL > > > > > > W0 > > > > > > OF > > > > > > > > > > > yo3wo3985rhAKPMLmmg6RA3V7I&m=3DHhHPjfcrk8XioEm16n75UIKYwi8c8= YrzV > > > > > > rp > > > > > > 5t > > > > > > BK > > > > > > 7L > > > > > > > > > > > X8&s=3DcJ2JGXAUQx4ymtMv_MLtGq7QiUJV3xBzKcS_Nwla08A&e=3D > > > > > > > > > > > Jira: > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__issue= s.ap > > > > > > ac > > > > > > he > > > > > > .o > > > > > > rg > > > > > > > > > > > _jira_browse_KAFKA-2D3932&d=3DDwIFaQ&c=3DZgVRmm3mf2P1-XDAyDs= u4A&r=3D > > > > > > Az > > > > > > 03 > > > > > > wM > > > > > > rb > > > > > > > > > > > L9ToLW0OFyo3wo3985rhAKPMLmmg6RA3V7I&m=3DHhHPjfcrk8XioEm16n75= UIKY > > > > > > wi > > > > > > 8c > > > > > > 8Y > > > > > > rz > > > > > > > > > > > Vrp5tBK7LX8&s=3DTfIIF2Ui9YEVxxwAbko0j-fT_mMVHf5Yywapc0w8eEA&= e=3D > > > > > > > > > > > > > > > > > > > > > > Thanks, CH > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > -Regards, > > > > Mayuresh R. Gharat > > > > (862) 250-7125 > > > > > > > > > > > > > -- > > > -Regards, > > > Mayuresh R. Gharat > > > (862) 250-7125 > > > > >=20 > >=20 > > -- > > -Regards, > > Mayuresh R. Gharat > > (862) 250-7125 > > Email had 1 attachment: > > + Re: [EXTERNAL] - Re: KAFKA-3932 - Consumer fails to consume in a=20= > > + round > > robin fashion > > 15k (message/rfc822) >