systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Boehm <mboe...@googlemail.com>
Subject Re: Release cadence
Date Thu, 23 Mar 2017 08:25:53 GMT
well, I think such a release is reasonable given that we focused so far
mostly on robustness features (parfor, OOMs, ultra-sparse) and various
fixes due to the uncertainty of the upcoming release. The new code
generator is also in reasonable shape to go out as an experimental feature.
Similarly, I don't see major issues caused by our move to Java 8 as well as
its exploitation via scalable counters, streams and lambda expressions.

With regard to backports of fixes into 0.14, however, let's decide on a
case-by-case basis as a general policy like that would create considerable
overhead.

Regards,
Matthias

On Wed, Mar 22, 2017 at 11:47 PM, Arvind Surve <acs_s@yahoo.com.invalid>
wrote:

> We have released SystemML 0.13, first code based on Spark 2.0.x, on March
> 2nd 2017.Some users may have transformed to SystemML 0.13 with some
> effort. Its not good idea to go out immediately with SystemML 1.0 as it has
> high potential to break backward compatibility.In that case, SystemML 0.13
> users who need to get later version/fix, they have to go with additional
> changes due to losing backward compatibility.
> To avoid immediate burden on current users I am proposing following.
> Lets cut SystemML 0.14 branch with current changes based on master
> branch.Then onward master branch can be used to do changes specific to
> SystemML 1.0 based code and any fixes based on current code.Whoever add any
> fix to SystemML 1.0 (master branch then) will need to put fixes on SystemML
> 0.14 branch as well as applicable to SystemML 0.13 code.Features added in
> SystemML 1.0 (master branch then) should not go into SystemML 0.14 branch.
> If there is no disagreement, then I will create new release SystemML 0.14
> on 03/30/2017. Please make sure to get your code in by EOD 03/29/2017.
> -Arvind Arvind Surve | Spark Technology Center  | http://www.spark.tc/
>
>       From: Berthold Reinwald <reinwald@us.ibm.com>
>  To: dev@systemml.incubator.apache.org
>  Sent: Sunday, March 12, 2017 5:54 PM
>  Subject: Re: Release cadence
>
> i am fine with 1.0 but let us stage it in a way that back porting of
> possible bug fixes will not be too difficult in the next few weeks or
> small nbr of month.
>
> Regards,
> Berthold Reinwald
> IBM Almaden Research Center
> office: (408) 927 2208; T/L: 457 2208
> e-mail: reinwald@us.ibm.com
>
>
>
> From:  Matthias Boehm <mboehm7@googlemail.com>
> To:    dev@systemml.incubator.apache.org
> Date:  03/12/2017 05:15 PM
> Subject:        Re: Release cadence
>
>
>
> ok great - as the majority is in favor of a 1.0 release and there is no
> veto, I think we came to an agreement here: our next release will be
> SystemML 1.0.
>
> Regards,
> Matthias
>
>
> On Tue, Mar 7, 2017 at 11:30 AM, <dusenberrymw@gmail.com> wrote:
>
> > +1 for immediately starting work on SystemML 1.0 as our next release.
> >
> > At this point, the project and our users will benefit most from a
> thorough
> > cleanup, as it will make the project simpler to use and easier to
> > maintain.  Simplicity will allow users and maintainers to regain focus
> on
> > ML research and products, which is a win for the entire community.  We
> > should create a solid list of items that we, and the rest of the
> community,
> > want to address for the 1.0 release and make sure that they are indeed
> > completed.  At the same time, we should ensure that we don't drag out
> the
> > release process.
> >
> > -Mike
> >
> > --
> >
> > Mike Dusenberry
> > GitHub: github.com/dusenberrymw
> > LinkedIn: linkedin.com/in/mikedusenberry
> >
> > Sent from my iPhone.
> >
> >
> > > On Mar 6, 2017, at 10:14 AM, Luciano Resende <luckbr1975@gmail.com>
> > wrote:
> > >
> > > +1 for SystemML 1.0 as the next release.
> > >
> > > On Sat, Mar 4, 2017 at 10:23 AM, Deron Eriksson
> <deroneriksson@gmail.com
> > >
> > > wrote:
> > >
> > >> Personally I would like the next release to be 1.0. We have been an
> > >> incubator project since November 2015 and I believe that after over
> > 1,000
> > >> commits since then that the project is about ready for a 1.0 release.
> > >>
> > >> I agree with Matthias that we need to make a decision regarding this
> > topic.
> > >> For new issues and fixed issues in JIRA, we need to be able to assign
> > the
> > >> correct version, or else someone potentially needs to go through and
> fix
> > >> the version numbers, as Glenn has been doing. Additionally, it would
> be
> > >> nice to do some of the 1.0 code updates (such as removing the old
> > >> MLContext) now rather than waiting additional months. Also I would
> like
> > to
> > >> be able to correctly identify our next version in the online
> > documentation.
> > >>
> > >>
> > > How about just make SystemML Next and change the release name when we
> do
> > > the release ?
> > >
> > >
> > >
> > >> Deron
> > >>
> > >>
> > >> On Sat, Mar 4, 2017 at 12:47 AM, Matthias Boehm
> <mboehm7@googlemail.com
> > >
> > >> wrote:
> > >>
> > >>> thanks Arvind for bringing some structure to the release process. I
> > >> think a
> > >>> fixed cadence of 2 months is useful as it makes upcoming releases
> more
> > >>> predictable for devs and users.
> > >>>
> > >>> However, we're discussing a major 1.0 release for a while now. I
> think
> > it
> > >>> would be useful to come to an agreement if we go for 1.0 in April or
> > not.
> > >>> There are some pending changes such as removing the old MLContext,
> > >> removing
> > >>> the file-based transform, isolating the matrix block library, and
> some
> > >>> language changes that should only be addresses in a major release as
> > they
> > >>> break backwards compatibility. Right now, we can't touch these
> changes
> > >>> without knowing the target release.
> > >>>
> > >>> Personally, I don't see a good reason why we should wait. Postponing
> > this
> > >>> major release just creates unnecessary overhead in maintaining these
> > old
> > >>> components that will be removed eventually. Since we cut RC for 0.13
> on
> > >> Feb
> > >>> 20, I think having an RC around April 20 would be a good target for
> > this
> > >>> 1.0 release.
> > >>>
> > >>>
> > >>> Regards,
> > >>> Matthias
> > >>>
> > >>>
> > >>> On Fri, Mar 3, 2017 at 5:44 PM, Arvind Surve
> <acs_s@yahoo.com.invalid>
> > >>> wrote:
> > >>>
> > >>>> Based on last couple of release cycles, we will continue with 2
> months
> > >>>> release cycles.We will do first RC build by end of first week of
> > second
> > >>>> month.
> > >>>> We will plan on releasing next release by end of April 2017.We
will
> > >> have
> > >>>> RC build on ~April 6th.  -Arvind
> > >>>> Arvind Surve | Spark Technology Center  | http://www.spark.tc/
> > >>>>
> > >>>>      From: Acs S <acs_s@yahoo.com.INVALID>
> > >>>> To: "dev@systemml.incubator.apache.org" <dev@systemml.incubator.
> > >>>> apache.org>
> > >>>> Sent: Monday, January 9, 2017 11:41 AM
> > >>>> Subject: Re: Release cadence
> > >>>>
> > >>>> We need to release SystemML on more frequent basis to get community
> > >>>> engaged. It will provide us more feedback on functionality we
> > add.While
> > >>>> releasing SystemML on monthly basis is challenge due to longer
> phase
> > of
> > >>>> validation process we need to find a way to be quicker.
> > >>>> I can propose options to get closer to monthly release if
> acceptable.
> > >>>> Make every two releases available on monthly basis and third on
two
> > >>> months
> > >>>> basis. This cycle will continue.
> > >>>> 1. Do minimal testing on two releases (minor releases) and release
> > them
> > >>> on
> > >>>> monthly basis. Performance testing is one of the major time
> consuming
> > >>>> activity especially for larger data size. We can limit testing
only
> > >> upto
> > >>>> 80GB. We can do code freeze (other than fixes) at the end of third
> > week
> > >>> and
> > >>>> do verification on last week. If we find any issues we can still
> > >> release
> > >>>> the code with limitation documented unless issue breaks major
> > >>>> functionality.2. Do comprehensive testing on third release.  This
> > will
> > >>>> include performance testing for all data size and every other
> testing
> > >> we
> > >>>> do. We can do code freeze (other than fixes) at the end of third
> week
> > >> and
> > >>>> start verification of code. All issues found will be addressed.
> > >> Exception
> > >>>> will be considered.
> > >>>>
> > >>>> Meantime we need to start automating testing pieces.
> > >>>>
> > >>>> Arvind SurveSpark Technology Centerhttp://www.spark.tc/
> > >>>>
> > >>>>      From: Berthold Reinwald <reinwald@us.ibm.com>
> > >>>> To: dev@systemml.incubator.apache.org
> > >>>> Sent: Saturday, January 7, 2017 1:35 AM
> > >>>> Subject: Re: Release cadence
> > >>>>
> > >>>> I think that a 2 month cycle would be a good compromise for
> > major/minor
> > >>>> releases. Fixpack release could be at a 1 month cycle.
> > >>>>
> > >>>>
> > >>>> Regards,
> > >>>> Berthold Reinwald
> > >>>> IBM Almaden Research Center
> > >>>> office: (408) 927 2208; T/L: 457 2208
> > >>>> e-mail: reinwald@us.ibm.com
> > >>>>
> > >>>>
> > >>>>
> > >>>> From:  Deron Eriksson <deroneriksson@gmail.com>
> > >>>> To:    dev@systemml.incubator.apache.org
> > >>>> Date:  01/05/2017 02:14 PM
> > >>>> Subject:        Re: Release cadence
> > >>>>
> > >>>>
> > >>>>
> > >>>> +1 for trying out a 1 month release cycle.
> > >>>>
> > >>>> However, I highly agree with Matthias that there is a lot of
> overhead
> > >>> with
> > >>>> releases, so it would be good if we can work to streamline/automate
> > the
> > >>>> process as much as possible. Also, it would be good to distribute
> the
> > >>>> tasks
> > >>>> around as much as possible. This can result in cross-training and
> help
> > >>>> avoid overburdening the same contributors each month.
> > >>>>
> > >>>> If the overhead slows us down too much, then we can go to a slower
> > >>> release
> > >>>> cycle.
> > >>>>
> > >>>> Deron
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>> On Thu, Jan 5, 2017 at 1:50 PM, <dusenberrymw@gmail.com>
wrote:
> > >>>>>
> > >>>>> +1 for adopting a 1 month release cycle.
> > >>>>>
> > >>>>> --
> > >>>>>
> > >>>>> Mike Dusenberry
> > >>>>> GitHub: github.com/dusenberrymw
> > >>>>> LinkedIn: linkedin.com/in/mikedusenberry
> > >>>>>
> > >>>>> Sent from my iPhone.
> > >>>>>
> > >>>>>
> > >>>>>> On Jan 5, 2017, at 1:35 PM, Luciano Resende
> <luckbr1975@gmail.com>
> > >>>>> wrote:
> > >>>>>>
> > >>>>>> On Thu, Jan 5, 2017 at 6:05 AM, Matthias Boehm
> > >>>> <mboehm7@googlemail.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> In general, I like the idea of aiming for consistent
release
> > >> cycles.
> > >>>>>>> However, every month is just too much, at least for
me. There is
> a
> > >>>>>>> considerable overhead associated with each release
for
> end-to-end
> > >>>>>>> performance tests, tests on different environments,
code freeze
> > >> for
> > >>>> new
> > >>>>>>> features, etc. Hence, a too short release cycle would
not be
> > >> "agile"
> > >>>> but
> > >>>>>>> would actually slow us down. From my perspective, a
realistic
> > >>> release
> > >>>>>>> cadence would be 2-3 months, maybe a bit more for major
> releases.
> > >>>>>>>
> > >>>>>>>
> > >>>>>> 2-3 months of release cadence for an open source is probably
a
> long
> > >>>>>> stretch, particular for a project that does not have very
large
> set
> > >>> of
> > >>>>> 3rd
> > >>>>>> party dependencies.
> > >>>>>>
> > >>>>>> As for some of the overhead issues you mentioned, they
are
> probably
> > >>>> easy
> > >>>>> to
> > >>>>>> workaround:
> > >>>>>>
> > >>>>>> - code-freeze timeframe can be resolved with branches
> > >>>>>> - end-to-end performance regressions can be avoided by
better
> code
> > >>>>> review,
> > >>>>>> and if you were willing to go with 2-3 months without performing
> > >>> these
> > >>>>>> tests, we could perform them only for major releases, and
> > >> proactively
> > >>>>>> quickly build a minor release with the patch when a user
report
> any
> > >>>>>> performance regression.
> > >>>>>>
> > >>>>>>
> > >>>>>> Anyway, I would really like to see SystemML more agile
with
> regards
> > >>> to
> > >>>>> its
> > >>>>>> release process because, as I mentioned before, the release
> early,
> > >>>>> release
> > >>>>>> often mantra is good to increase community interest, generate
> more
> > >>>>> traffic
> > >>>>>> to the list as developers discuss the roadmap and release
> blockers,
> > >>>> and
> > >>>>>> also enable users to provide feedback sooner on the areas
we are
> > >>>>> developing.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Luciano Resende
> > >>>>>> http://twitter.com/lresende1975
> > >>>>>> http://lresende.blogspot.com/
> > >>>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Deron Eriksson
> > >>>> Spark Technology Center
> > >>>> http://www.spark.tc/
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Deron Eriksson
> > >> Spark Technology Center
> > >> http://www.spark.tc/
> > >>
> > >
> > >
> > >
> > > --
> > > Luciano Resende
> > > http://twitter.com/lresende1975
> > > http://lresende.blogspot.com/
> >
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message