kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: [VOTE] KIP-345: Introduce static membership protocol to reduce consumer rebalances
Date Mon, 03 Dec 2018 17:27:25 GMT
Hello Boyang,

I've browsed through the new wiki and there are still a couple of minor
things to notice:

1. RemoveMemberFromGroupOptions seems not defined anywhere.

2. LeaveGroupRequest added a list of group instance id, but still keep the
member id as a singleton; is that intentional? I think to make the protocol
consistent both member id and instance ids could be plural.

3. About the *kafka-remove-member-from-group.sh *tool, I'm wondering if we
can defer adding this while just add the corresponding calls of the
LeaveGroupRequest inside Streams until we have used it in production and
hence have a better understanding on how flexible or extensible if we want
to add any cmd tools. The rationale is that if we do not necessarily need
it now, we can always add it later with a more think-through API design,
but if we add the tool in a rush, we may need to extend or modify it soon
after we realize its limits in operations.

Otherwise, I'm +1 on the proposal.


On Mon, Dec 3, 2018 at 9:14 AM Boyang Chen <bchen11@outlook.com> wrote:

> Hey community friends,
> after another month of polishing, KIP-345<
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances>
> design is ready for vote. Feel free to add your comment on the discussion
> thread or here.
> Thanks for your time!
> Boyang
> ________________________________
> From: Boyang Chen <bchen11@outlook.com>
> Sent: Friday, November 9, 2018 6:35 AM
> To: dev@kafka.apache.org
> Subject: [VOTE] KIP-345: Introduce static membership protocol to reduce
> consumer rebalances
> Hey all,
> thanks so much for all the inputs on KIP-345 so far. The original proposal
> has enhanced a lot with your help. To make sure the implementation go
> smoothly without back and forth, I would like to start a vote on the final
> design agreement now:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-<
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances
> >
> 345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances<
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances
> >
> KIP-345: Introduce static membership protocol to reduce ...<
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances
> >
> cwiki.apache.org
> For stateful applications, one of the biggest performance bottleneck is
> the state shuffling. In Kafka consumer, there is a concept called
> "rebalance" which means that for given M partitions and N consumers in one
> consumer group, Kafka will try to balance the load between consumers and
> ideally have ...
> Let me know if you have any questions.
> Best,
> Boyang

-- Guozhang

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message