kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Yu <yohan.richard...@gmail.com>
Subject Re: [DISCUSS] KIP-262 Metadata should include the number of state stores for task
Date Thu, 03 Jan 2019 21:41:56 GMT
Hi Matthias,

I guess this is no longer necessary. I am open to anything honestly.
I suppose we should close it (if its not already).


On Fri, Oct 19, 2018 at 11:06 AM Matthias J. Sax <matthias@confluent.io>
wrote:

> Any thought on my last email about discarding this KIP?
>
>
> -Matthias
>
> On 9/14/18 11:44 AM, Matthias J. Sax wrote:
> > Hi,
> >
> > we recently had a discussion on a different ticket to reduce the size of
> > the metadata we need to send:
> > https://issues.apache.org/jira/browse/KAFKA-7149
> >
> > It seems, that we actually don't need to include the number of stores in
> > the metadata, but that we can compute the number of stores locally on
> > each instance.
> >
> > With this insight, we should still try to exploit this knowledge during
> > task assignment, however, this would be an internal change that does not
> > require a KIP. Thus, I think that we can discard this KIP.
> >
> > Thoughts?
> >
> >
> > -Matthias
> >
> > On 6/10/18 5:20 PM, Matthias J. Sax wrote:
> >> Richard,
> >>
> >> KIP-268 got merged and thus this KIP is unblocked.
> >>
> >> I just re-read it and think it needs some updates with regard to the
> >> upgrade path (ie, you should mention why upgrading is covered).
> >>
> >> It would also be useful to discuss how the store information is used
> >> during assignment. Atm, the KIP only discussed that the information
> >> should be added, but this is only half of the story from my point of
> view.
> >>
> >>
> >> -Matthias
> >>
> >> On 3/22/18 9:15 PM, Matthias J. Sax wrote:
> >>> Hi Richard,
> >>>
> >>> with KIP-268 in place (should be accepted soon) the upgrade path is
> >>> covered. Thus, you can update your KIP accordingly, referring to
> KIP-268.
> >>>
> >>> Can you also update your KIP similar to KIP-268 to cover the old and
> new
> >>> metadata format?
> >>>
> >>> Thanks!
> >>>
> >>> -Matthias
> >>>
> >>>
> >>> On 2/24/18 4:07 PM, Richard Yu wrote:
> >>>> I didn't really get what "upgrade strategy" was at the time that
> Guozhang
> >>>> mentioned it, so I wrote the above dialogue from my first
> understanding. I
> >>>> changed it to "upgrade strategy must be provided". Currently,
> however, I do
> >>>> not have anything in mind to facilitate upgrading older Kafka
> brokers. If
> >>>> you have anything in mind, please let me know.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Sat, Feb 24, 2018 at 3:58 PM, Matthias J. Sax <
> matthias@confluent.io>
> >>>> wrote:
> >>>>
> >>>>> Thanks a lot for this KIP.
> >>>>>
> >>>>> I am not sure what you mean by
> >>>>>
> >>>>>> which could potentially break older versions of Kafka brokers
> >>>>>
> >>>>> The metadata that is exchange, is not interpreted by the brokers.
The
> >>>>> problem with upgrading the metadata format affect only Kafka Streams
> >>>>> instances.
> >>>>>
> >>>>> If we don't provide an upgrade strategy, changing the metadata format
> >>>>> required to stop all running application instances, before the
> instances
> >>>>> can be restarted with the new code. However, this implies downtime
> for
> >>>>> an application and is thus not acceptable.
> >>>>>
> >>>>>
> >>>>> -Matthias
> >>>>>
> >>>>>
> >>>>> On 2/24/18 11:11 AM, Richard Yu wrote:
> >>>>>> Hi all,
> >>>>>>
> >>>>>> I would like to discuss a KIP I've submitted :
> >>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>>>> 262%3A+Metadata+should+include+number+of+state+stores+for+task
> >>>>>>
> >>>>>>
> >>>>>> Regards,
> >>>>>> Richard Yu
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message