systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arvind Surve <ac...@yahoo.com.INVALID>
Subject Re: Release cadence
Date Thu, 23 Mar 2017 06:47:12 GMT
We have released SystemML 0.13, first code based on Spark 2.0.x, on March 2nd 2017.Some users
may have transformed to SystemML 0.13 with some effort. Its not good idea to go out immediately
with SystemML 1.0 as it has high potential to break backward compatibility.In that case, SystemML
0.13 users who need to get later version/fix, they have to go with additional changes due
to losing backward compatibility.
To avoid immediate burden on current users I am proposing following.
Lets cut SystemML 0.14 branch with current changes based on master branch.Then onward master
branch can be used to do changes specific to SystemML 1.0 based code and any fixes based on
current code.Whoever add any fix to SystemML 1.0 (master branch then) will need to put fixes
on SystemML 0.14 branch as well as applicable to SystemML 0.13 code.Features added in SystemML
1.0 (master branch then) should not go into SystemML 0.14 branch.
If there is no disagreement, then I will create new release SystemML 0.14 on 03/30/2017. Please
make sure to get your code in by EOD 03/29/2017.
-Arvind Arvind Surve | Spark Technology Center  | http://www.spark.tc/

      From: Berthold Reinwald <reinwald@us.ibm.com>
 To: dev@systemml.incubator.apache.org 
 Sent: Sunday, March 12, 2017 5:54 PM
 Subject: Re: Release cadence
   
i am fine with 1.0 but let us stage it in a way that back porting of 
possible bug fixes will not be too difficult in the next few weeks or 
small nbr of month.

Regards,
Berthold Reinwald
IBM Almaden Research Center
office: (408) 927 2208; T/L: 457 2208
e-mail: reinwald@us.ibm.com



From:  Matthias Boehm <mboehm7@googlemail.com>
To:    dev@systemml.incubator.apache.org
Date:  03/12/2017 05:15 PM
Subject:        Re: Release cadence



ok great - as the majority is in favor of a 1.0 release and there is no
veto, I think we came to an agreement here: our next release will be
SystemML 1.0.

Regards,
Matthias


On Tue, Mar 7, 2017 at 11:30 AM, <dusenberrymw@gmail.com> wrote:

> +1 for immediately starting work on SystemML 1.0 as our next release.
>
> At this point, the project and our users will benefit most from a 
thorough
> cleanup, as it will make the project simpler to use and easier to
> maintain.  Simplicity will allow users and maintainers to regain focus 
on
> ML research and products, which is a win for the entire community.  We
> should create a solid list of items that we, and the rest of the 
community,
> want to address for the 1.0 release and make sure that they are indeed
> completed.  At the same time, we should ensure that we don't drag out 
the
> release process.
>
> -Mike
>
> --
>
> Mike Dusenberry
> GitHub: github.com/dusenberrymw
> LinkedIn: linkedin.com/in/mikedusenberry
>
> Sent from my iPhone.
>
>
> > On Mar 6, 2017, at 10:14 AM, Luciano Resende <luckbr1975@gmail.com>
> wrote:
> >
> > +1 for SystemML 1.0 as the next release.
> >
> > On Sat, Mar 4, 2017 at 10:23 AM, Deron Eriksson 
<deroneriksson@gmail.com
> >
> > wrote:
> >
> >> Personally I would like the next release to be 1.0. We have been an
> >> incubator project since November 2015 and I believe that after over
> 1,000
> >> commits since then that the project is about ready for a 1.0 release.
> >>
> >> I agree with Matthias that we need to make a decision regarding this
> topic.
> >> For new issues and fixed issues in JIRA, we need to be able to assign
> the
> >> correct version, or else someone potentially needs to go through and 
fix
> >> the version numbers, as Glenn has been doing. Additionally, it would 
be
> >> nice to do some of the 1.0 code updates (such as removing the old
> >> MLContext) now rather than waiting additional months. Also I would 
like
> to
> >> be able to correctly identify our next version in the online
> documentation.
> >>
> >>
> > How about just make SystemML Next and change the release name when we 
do
> > the release ?
> >
> >
> >
> >> Deron
> >>
> >>
> >> On Sat, Mar 4, 2017 at 12:47 AM, Matthias Boehm 
<mboehm7@googlemail.com
> >
> >> wrote:
> >>
> >>> thanks Arvind for bringing some structure to the release process. I
> >> think a
> >>> fixed cadence of 2 months is useful as it makes upcoming releases 
more
> >>> predictable for devs and users.
> >>>
> >>> However, we're discussing a major 1.0 release for a while now. I 
think
> it
> >>> would be useful to come to an agreement if we go for 1.0 in April or
> not.
> >>> There are some pending changes such as removing the old MLContext,
> >> removing
> >>> the file-based transform, isolating the matrix block library, and 
some
> >>> language changes that should only be addresses in a major release as
> they
> >>> break backwards compatibility. Right now, we can't touch these 
changes
> >>> without knowing the target release.
> >>>
> >>> Personally, I don't see a good reason why we should wait. Postponing
> this
> >>> major release just creates unnecessary overhead in maintaining these
> old
> >>> components that will be removed eventually. Since we cut RC for 0.13 
on
> >> Feb
> >>> 20, I think having an RC around April 20 would be a good target for
> this
> >>> 1.0 release.
> >>>
> >>>
> >>> Regards,
> >>> Matthias
> >>>
> >>>
> >>> On Fri, Mar 3, 2017 at 5:44 PM, Arvind Surve 
<acs_s@yahoo.com.invalid>
> >>> wrote:
> >>>
> >>>> Based on last couple of release cycles, we will continue with 2 
months
> >>>> release cycles.We will do first RC build by end of first week of
> second
> >>>> month.
> >>>> We will plan on releasing next release by end of April 2017.We will
> >> have
> >>>> RC build on ~April 6th.  -Arvind
> >>>> Arvind Surve | Spark Technology Center  | http://www.spark.tc/
> >>>>
> >>>>      From: Acs S <acs_s@yahoo.com.INVALID>
> >>>> To: "dev@systemml.incubator.apache.org" <dev@systemml.incubator.
> >>>> apache.org>
> >>>> Sent: Monday, January 9, 2017 11:41 AM
> >>>> Subject: Re: Release cadence
> >>>>
> >>>> We need to release SystemML on more frequent basis to get community
> >>>> engaged. It will provide us more feedback on functionality we
> add.While
> >>>> releasing SystemML on monthly basis is challenge due to longer 
phase
> of
> >>>> validation process we need to find a way to be quicker.
> >>>> I can propose options to get closer to monthly release if 
acceptable.
> >>>> Make every two releases available on monthly basis and third on two
> >>> months
> >>>> basis. This cycle will continue.
> >>>> 1. Do minimal testing on two releases (minor releases) and release
> them
> >>> on
> >>>> monthly basis. Performance testing is one of the major time 
consuming
> >>>> activity especially for larger data size. We can limit testing only
> >> upto
> >>>> 80GB. We can do code freeze (other than fixes) at the end of third
> week
> >>> and
> >>>> do verification on last week. If we find any issues we can still
> >> release
> >>>> the code with limitation documented unless issue breaks major
> >>>> functionality.2. Do comprehensive testing on third release.  This
> will
> >>>> include performance testing for all data size and every other 
testing
> >> we
> >>>> do. We can do code freeze (other than fixes) at the end of third 
week
> >> and
> >>>> start verification of code. All issues found will be addressed.
> >> Exception
> >>>> will be considered.
> >>>>
> >>>> Meantime we need to start automating testing pieces.
> >>>>
> >>>> Arvind SurveSpark Technology Centerhttp://www.spark.tc/
> >>>>
> >>>>      From: Berthold Reinwald <reinwald@us.ibm.com>
> >>>> To: dev@systemml.incubator.apache.org
> >>>> Sent: Saturday, January 7, 2017 1:35 AM
> >>>> Subject: Re: Release cadence
> >>>>
> >>>> I think that a 2 month cycle would be a good compromise for
> major/minor
> >>>> releases. Fixpack release could be at a 1 month cycle.
> >>>>
> >>>>
> >>>> Regards,
> >>>> Berthold Reinwald
> >>>> IBM Almaden Research Center
> >>>> office: (408) 927 2208; T/L: 457 2208
> >>>> e-mail: reinwald@us.ibm.com
> >>>>
> >>>>
> >>>>
> >>>> From:  Deron Eriksson <deroneriksson@gmail.com>
> >>>> To:    dev@systemml.incubator.apache.org
> >>>> Date:  01/05/2017 02:14 PM
> >>>> Subject:        Re: Release cadence
> >>>>
> >>>>
> >>>>
> >>>> +1 for trying out a 1 month release cycle.
> >>>>
> >>>> However, I highly agree with Matthias that there is a lot of 
overhead
> >>> with
> >>>> releases, so it would be good if we can work to streamline/automate
> the
> >>>> process as much as possible. Also, it would be good to distribute 
the
> >>>> tasks
> >>>> around as much as possible. This can result in cross-training and 
help
> >>>> avoid overburdening the same contributors each month.
> >>>>
> >>>> If the overhead slows us down too much, then we can go to a slower
> >>> release
> >>>> cycle.
> >>>>
> >>>> Deron
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>> On Thu, Jan 5, 2017 at 1:50 PM, <dusenberrymw@gmail.com> wrote:
> >>>>>
> >>>>> +1 for adopting a 1 month release cycle.
> >>>>>
> >>>>> --
> >>>>>
> >>>>> Mike Dusenberry
> >>>>> GitHub: github.com/dusenberrymw
> >>>>> LinkedIn: linkedin.com/in/mikedusenberry
> >>>>>
> >>>>> Sent from my iPhone.
> >>>>>
> >>>>>
> >>>>>> On Jan 5, 2017, at 1:35 PM, Luciano Resende 
<luckbr1975@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> On Thu, Jan 5, 2017 at 6:05 AM, Matthias Boehm
> >>>> <mboehm7@googlemail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> In general, I like the idea of aiming for consistent release
> >> cycles.
> >>>>>>> However, every month is just too much, at least for me.
There is 
a
> >>>>>>> considerable overhead associated with each release for 
end-to-end
> >>>>>>> performance tests, tests on different environments, code
freeze
> >> for
> >>>> new
> >>>>>>> features, etc. Hence, a too short release cycle would not
be
> >> "agile"
> >>>> but
> >>>>>>> would actually slow us down. From my perspective, a realistic
> >>> release
> >>>>>>> cadence would be 2-3 months, maybe a bit more for major

releases.
> >>>>>>>
> >>>>>>>
> >>>>>> 2-3 months of release cadence for an open source is probably
a 
long
> >>>>>> stretch, particular for a project that does not have very large

set
> >>> of
> >>>>> 3rd
> >>>>>> party dependencies.
> >>>>>>
> >>>>>> As for some of the overhead issues you mentioned, they are 
probably
> >>>> easy
> >>>>> to
> >>>>>> workaround:
> >>>>>>
> >>>>>> - code-freeze timeframe can be resolved with branches
> >>>>>> - end-to-end performance regressions can be avoided by better

code
> >>>>> review,
> >>>>>> and if you were willing to go with 2-3 months without performing
> >>> these
> >>>>>> tests, we could perform them only for major releases, and
> >> proactively
> >>>>>> quickly build a minor release with the patch when a user report

any
> >>>>>> performance regression.
> >>>>>>
> >>>>>>
> >>>>>> Anyway, I would really like to see SystemML more agile with

regards
> >>> to
> >>>>> its
> >>>>>> release process because, as I mentioned before, the release

early,
> >>>>> release
> >>>>>> often mantra is good to increase community interest, generate

more
> >>>>> traffic
> >>>>>> to the list as developers discuss the roadmap and release 
blockers,
> >>>> and
> >>>>>> also enable users to provide feedback sooner on the areas we
are
> >>>>> developing.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Luciano Resende
> >>>>>> http://twitter.com/lresende1975
> >>>>>> http://lresende.blogspot.com/
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Deron Eriksson
> >>>> Spark Technology Center
> >>>> http://www.spark.tc/
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> Deron Eriksson
> >> Spark Technology Center
> >> http://www.spark.tc/
> >>
> >
> >
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
>





   
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message