flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: [DISCUSS] Time-based releases in Flink
Date Mon, 30 Jan 2017 15:03:54 GMT
Thanks again for all your feedback on my proposal!

The discussion was open for the last 12 days and the majority was very
positive about it.
I will keep an eye on the valid concerns Greg raised (neglected PRs,
unstable releases) and see how we can solve them.

I'll update the wiki pages and include a schedule for upcoming releases.
I'll also try to send reminders to the mailing list about upcoming release
related deadlines.

On Thu, Jan 19, 2017 at 7:15 PM, Robert Metzger <rmetzger@apache.org> wrote:

> Thanks a lot for your response Greg!
>
> I'd like to hear a retrospective on the duration of the 1.2 release cycle.
>
>
> 164 days, or 5 months and 11 days have passed since the 1.1 release. The
> other release cycles are listed here: https://cwiki.apache.
> org/confluence/display/FLINK/Flink+Release+and+Feature+Plan
> I would like to increase the release frequency a bit to 4 months, because
> I'm often seeing a lot of interest from our users for the latest features.
> The stream processing space is still quickly evolving, so I would like to
> bring the latest new features out as fast as possible (without compromising
> quality of course).
>
> As noted recently within the Flink community, the number of outstanding
>> pull requests and tickets has steadily increased. I'm concerned that with
>> fixed deadlines committer priorities will take precedence and community
>> contributions will be deferred indefinitely.
>
>
> You are raising a very valid point: Community work hasn't been as good as
> it was a few months ago, and we should fix that as soon as possible.
>
> However, I don't think that time-based releases will change much on that
> situation. Both models can potentially draw all the attention to the
> release deadline vs working on community contributions.
> If you look back at the emails regarding the 1.2.0 release, it was first
> brought up by Fabian in mid October. Early December, I was starting to
> track the progress on the release on the ML. From that time (almost two
> months ago) the committers were focusing a lot on getting their stuff into
> the release. I think that's part of the reason why the community work
> hasn't been that great recently.
>
> Maybe it would make sense to start a separate thread to discuss how we can
> fix the situation with the number of open pull requests?
>
>
> I'm concerned that only blockers are fixed for a .0 release, and that
>> exactly two weeks are allotted. Will any desirable fix simply become a
>> blocker or will we potentially release with a list of known bugs?
>
>
> I hope neither will happen too much, if we enforce that big features don't
> get merged in the last minute and thus get enough testing exposure before
> the merge-window closes.
> I have to admit that I don't have a lot of experience with a strictly
> time-based release model, but we should definitively review the process
> after the 1.3.0 release (and of course ensure that the master is getting
> stable before the release branch is forked off)
>
> The good thing about the time-based release model is, that everybody knows
> well in advance what the dates for feature freeze and code freeze are.
> So the code freeze should not come as a surprise and people should test
> and fix their stuff before that happens.
>
>
> Regarding JIRA: I agree that it is not very well maintained right now, and
> that the "fix version" flag is used more as a wish than a plan.
> I will not move all the 1.2 issues that haven't been closed to 1.3 to
> avoid having a bad start for the time-based releases.
>
>
>
>
>
>
>
> On Wed, Jan 18, 2017 at 5:34 PM, Greg Hogan <code@greghogan.com> wrote:
>
>> I'm +0 on switching to a pre-determined schedule. It may be that the Flink
>> codebase has reached a level of maturity allowing for a time-based release
>> schedule, and I'm hopeful that a known schedule will improve communication
>> about and expectations for new features.
>>
>> I'd like to hear a retrospective on the duration of the 1.2 release cycle.
>>
>> As noted recently within the Flink community, the number of outstanding
>> pull requests and tickets has steadily increased. I'm concerned that with
>> fixed deadlines committer priorities will take precedence and community
>> contributions will be deferred indefinitely.
>>
>> I'm concerned that only blockers are fixed for a .0 release, and that
>> exactly two weeks are allotted. Will any desirable fix simply become a
>> blocker or will we potentially release with a list of known bugs?
>>
>> I think it would be worthwhile to identity upgradeable dependencies at the
>> beginning of each release cycle which would allow the most time for
>> testing
>> during development.
>>
>> There are 130 open tickets scheduled for 1.2.0 [1]. Targeting "inbox zero"
>> (without simply bulk-migrating tickets to another release) would be
>> preferable to tracking tickets through email chains on the mailing list.
>> The number of open GitHub pull requests isn't so much as issue since PRs
>> are simply listed by date, but it would be helpful to keep Jira tickets
>> up-to-date. It seems that "Fix Version" is often a wish rather than a
>> plan.
>>
>> Greg
>>
>> [1]
>> https://issues.apache.org/jira/browse/FLINK?selectedTab=com.
>> atlassian.jira.jira-projects-plugin:issues-panel
>>
>> On Wed, Jan 18, 2017 at 5:13 AM, Robert Metzger <rmetzger@apache.org>
>> wrote:
>>
>> > Thanks a lot for the positive feedback so far.
>> >
>> > Thank you Fabian for spotting the off by one error in my email.
>> > "There are two hard things in computer science: cache invalidation,
>> naming
>> > things, and off-by-one errors." (https://twitter.com/codinghor
>> ror/status/
>> > 506010907021828096?lang=en)
>> >
>> > I agree with Fabian that this model could potentially lead to more
>> feature
>> > branches in our repository. However, I think we should only do that for
>> > major features where many contributors are collaborating on. Using
>> private
>> > development branches works equally well for smaller groups.
>> > I found that the frequent "flip6" branch rebases caused a lot of noise
>> on
>> > the commits@flink list.
>> >
>> > Regarding the "bugfix guarantees":
>> > I agree that my proposal doesn't make so much sense. I actually got the
>> > same feedback from a draft of my email but forgot to change it before
>> > sending it out. I think supporting the current (1.1) and previous (1.0)
>> > major release is a doable guarantee.
>> >
>> >
>> >
>> >
>> > On Wed, Jan 18, 2017 at 10:25 AM, Vasiliki Kalavri <
>> > vasilikikalavri@gmail.com> wrote:
>> >
>> > > Hi Robert,
>> > >
>> > > thanks a lot for starting this discussion and for putting together the
>> > wiki
>> > > pages.
>> > > This proposal makes a lot of sense to me.
>> > >
>> > > Big +1 for merging only features which are tested and *documented*.
>> > >
>> > > I believe that having a clear timeline will not only make users
>> happier
>> > but
>> > > also contributors more engaged. With the current unpredictability, it
>> is
>> > > hard to book time aside to help with testing a release candidate. I
>> hope
>> > > that knowing exactly when that needs to happen will help us plan our
>> own
>> > > time better and help out more.
>> > >
>> > > Cheers,
>> > > -Vasia.
>> > >
>> > > On 18 January 2017 at 09:57, Tzu-Li (Gordon) Tai <tzulitai@apache.org
>> >
>> > > wrote:
>> > >
>> > > > Hi Robert,
>> > > >
>> > > > Thanks for bringing up the discussion. I like the proposal.
>> > > >
>> > > > Regarding some of the downsides mentioned in the wiki:
>> > > >
>> > > > 1. Features that don’t make it in time with the feature freeze:
>> > > > I think that’s ok, as long as we’re consistent with the schedules
>> for
>> > the
>> > > > next release. This way users waiting for that particular features
>> will
>> > > > still be able to rely on the fact that the feature will be included
>> in
>> > 4
>> > > > months.
>> > > >
>> > > > 2. Frequent releases mean bug fix releases for older branches:
>> > > > You mentioned in the email that “old releases are supported for
6
>> > months
>> > > > by the community”, but not in the wiki. If this is strictly
>> followed,
>> > > that
>> > > > means we’ll at most be supporting 2 previous major release versions
>> > (ex.
>> > > as
>> > > > soon as 1.4.0 comes out, we’ll still be supporting bugfixes for
>> 1.3.0,
>> > as
>> > > > well as 1.2.0 for another 2 months).
>> > > > This might seem a bit odd, so perhaps we can stick to something like
>> > > > “support bugfixes for the previous 2 major releases”? Ex. Once
1.4.0
>> > > comes
>> > > > out, we’ll continue to support only 1.4.0 and 1.3.0.
>> > > > Supporting bugfixes for 2 major versions seems workable, and this
>> way
>> > > > users can also have a “buffer” that they should not fall behind
>> > releases
>> > > > for more than 2 major versions (8 months) and preplan upgrades.
>> > > >
>> > > > - Gordon
>> > > >
>> > > > On January 18, 2017 at 9:19:41 AM, Robert Metzger (
>> rmetzger@apache.org
>> > )
>> > > > wrote:
>> > > >
>> > > > Hi all!
>> > > >
>> > > > Since the 1.2.0 release is about to come out, I would like to
>> propose a
>> > > > change in the way we do releases in the Flink community.
>> > > >
>> > > > In my opinion, the current model leads to dissatisfaction among
>> users
>> > and
>> > > > contributors, because releases are really not predictable. A recent
>> > > example
>> > > > for the issues with our current model are the FLIP-6 changes we
>> wanted
>> > to
>> > > > merge to master, a few weeks before the first RC for 1.2.0. Also,
>> there
>> > > > were some emails on the mailing lists asking for a release date.
>> > > >
>> > > > In order to change that, I’m proposing to follow a strictly
>> time-based
>> > > > release model. Other open source projects like Ubuntu, Cassandra,
>> Spark
>> > > or
>> > > > Kafka are following that model as well, and I think we should try
it
>> > out
>> > > as
>> > > > an experiment for the 1.3.0 release.
>> > > >
>> > > > I’m proposing to:
>> > > >
>> > > > -
>> > > >
>> > > > Do a Flink release every 4 months
>> > > > -
>> > > >
>> > > > Cycle:
>> > > > -
>> > > >
>> > > > 3 months development
>> > > > -
>> > > >
>> > > > 1 month before the release: Feature freeze. Create release branch
>> > > > with RC0, start testing. Only fixes, tests and minor improvements
>> are
>> > > > allowed in.
>> > > > -
>> > > >
>> > > > 2 weeks before the release: Code freeze. Start voting. Only fixes
>> for
>> > > > blockers are allowed in.
>> > > > -
>> > > >
>> > > > Forbid blocking a release because a feature is not done yet.
>> > > > -
>> > > >
>> > > > Features are merged to master, when they are tested and documented,
>> to
>> > > > have an always stable master
>> > > > -
>> > > >
>> > > > Bugfix releases are done as needed.
>> > > > -
>> > > >
>> > > > Old releases are supported for 6 months by the Flink community with
>> > > > critical bug fixes
>> > > >
>> > > >
>> > > > This means, that we would have the following release dates:
>> > > >
>> > > > (Flink 1.3.0 by end of January 2017)
>> > > >
>> > > > Flink 1.4.0 by end of May 2017
>> > > >
>> > > > Flink 1.5.0 by end of September 2017
>> > > >
>> > > > Flink 1.6.0 by end of January 2018
>> > > >
>> > > > I’ve put some more details including some pro’s and con’s into
our
>> > wiki.
>> > > > The page is based on Kafka’s time-based release wiki page (Kafka
>> also
>> > > > recently started following a strictly time-based model)
>> > > >
>> > > > https://cwiki.apache.org/confluence/display/FLINK/Time-based
>> +releases
>> > > >
>> > > >
>> > > > Once we’ve agreed on following that model, I’ll update the release
>> plan
>> > > > page accordingly:
>> > > > https://cwiki.apache.org/confluence/display/FLINK/
>> > > > Flink+Release+and+Feature+Plan
>> > > >
>> > > >
>> > > > Please let me know what you think about this idea!
>> > > >
>> > > > Regards,
>> > > >
>> > > > Robert
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message