kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams
Date Mon, 04 May 2020 17:00:17 GMT
Thanks for the explanation John.


Guozhang

On Sun, May 3, 2020 at 7:10 PM John Roesler <vvcephei@apache.org> wrote:

> Hi Guozhang,
>
> Ah, good question. Yes, the assignor will always now try to achieve a
> perfect balance. This was also the proposed default in the KIP before. The
> config would have allowed users to relax the search for perfection.
>
> This is actually one of our motivations now to remove it. We feel it’s
> simpler to reason about the behavior of the system if you know it’s always
> going to produce a balanced assignment.
>
> Thanks,
> John
>
> On Sun, May 3, 2020, at 19:03, Guozhang Wang wrote:
> > Hello John / Sophie:
> >
> > With this config removed, would the assignor always tries to to achieve
> the
> > "perfect balance" (of course, it may be a sub-optimal local plateau) or
> > there's an internal hard-coded factor to still retain some satisfying
> > threshold?
> >
> > Guozhang
> >
> > On Sun, May 3, 2020 at 9:23 AM John Roesler <vvcephei@apache.org> wrote:
> >
> > > Hi Matthias,
> > >
> > > We originally proposed that config to allow us to skip migrating tasks
> if
> > > the current balance is “good enough”. But during implementation, we
> became
> > > concerned that supporting this option increased code complexity, and
> it’s
> > > also an extra concept for users to have to learn.
> > >
> > > To keep the new balancing system simpler both internally and
> externally,
> > > we’d like to drop it from the API for now, with the idea of adding it
> later
> > > if needed.
> > >
> > > Does that seem reasonable?
> > >
> > > Thanks,
> > > John
> > >
> > > On Fri, May 1, 2020, at 14:18, Matthias J. Sax wrote:
> > > > Can you elaborate why to remove it?
> > > >
> > > > On 5/1/20 11:29 AM, Sophie Blee-Goldman wrote:
> > > > > Hey all,
> > > > >
> > > > > We'd like to make a slight modification to the proposal in this
> KIP and
> > > > > remove
> > > > > the *balance.factor* config. We will update the KIP accordingly.
> > > Please let
> > > > > us know
> > > > > if you have any concerns.
> > > > >
> > > > > Cheers,
> > > > > Sophie
> > > > >
> > > > > On Wed, Jan 15, 2020 at 12:48 PM John Roesler <vvcephei@apache.org
> >
> > > wrote:
> > > > >
> > > > >> Hello all,
> > > > >>
> > > > >> After a long hiatus, I've just realized that I'm now able to
> upgrade
> > > my
> > > > >> non-binding support to a binding +1 for KIP-441.
> > > > >>
> > > > >> This brings the vote tally to:
> > > > >> 3 binding +1s: Guozhang, Bill, and myself
> > > > >> 3 non-binding +1s: Bruno, Vinoth, and Sophie
> > > > >>
> > > > >> Since the vote has been open for at least 72 hours, the KIP is
> > > accepted.
> > > > >>
> > > > >> Thanks all,
> > > > >> -John
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Mon, Oct 28, 2019 at 21:02 PM John Roesler <john@confluent.io>
> > > wrote:
> > > > >>> Hey all,
> > > > >>>
> > > > >>> Now that the 2.4 release storm is over, I'd like to bump
this
> vote
> > > > >> thread.
> > > > >>>
> > > > >>> Currently, we have two binding +1s (Guozhang and Bill), and
four
> > > > >>> non-binding ones (Bruno, Vinoth, Sophie, and myself), and
no
> vetoes.
> > > > >>>
> > > > >>> Thanks,
> > > > >>> -John
> > > > >>>
> > > > >>> On Thu, Sep 12, 2019 at 12:54 PM Bill Bejeck <bbejeck@gmail.com>
> > > wrote:
> > > > >>>>
> > > > >>>> +1 (binding)
> > > > >>>>
> > > > >>>> On Thu, Sep 12, 2019 at 1:53 PM Sophie Blee-Goldman <
> > > > >> sophie@confluent.io> wrote:
> > > > >>>>
> > > > >>>>> +1 (non-binding)
> > > > >>>>>
> > > > >>>>> On Wed, Sep 11, 2019 at 11:38 AM Vinoth Chandar <
> > > > >> vchandar@confluent.io> wrote:
> > > > >>>>>
> > > > >>>>>> +1 (non-binding).
> > > > >>>>>>
> > > > >>>>>> On Fri, Sep 6, 2019 at 12:46 AM Bruno Cadonna
<
> bruno@confluent.io
> > > >
> > > > >> wrote:
> > > > >>>>>>
> > > > >>>>>>> +1 (non-binding)
> > > > >>>>>>>
> > > > >>>>>>> On Fri, Sep 6, 2019 at 12:32 AM Guozhang
Wang <
> > > > >> wangguoz@gmail.com> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>> +1 (binding).
> > > > >>>>>>>>
> > > > >>>>>>>> On Thu, Sep 5, 2019 at 2:47 PM John Roesler
<
> john@confluent.io>
> > > > >> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>> Hello, all,
> > > > >>>>>>>>>
> > > > >>>>>>>>> After a great discussion, I'd like
to open voting on
> KIP-441,
> > > > >>>>>>>>> to avoid long restore times in Streams
after rebalancing.
> > > > >>>>>>>>> Please cast your votes!
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-441:+Smooth+Scaling+Out+for+Kafka+Streams
> > > > >>>>>>>>>
> > > > >>>>>>>>> Thanks,
> > > > >>>>>>>>> -John
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> --
> > > > >>>>>>>> -- Guozhang
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>
> > > > >>
> > > > >
> > > >
> > > >
> > > > Attachments:
> > > > * signature.asc
> > >
> >
> >
> > --
> > -- Guozhang
> >
>


-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message