Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (athena.apache.org: domain of rohityadav89@gmail.com
 designates 74.125.83.52 as permitted sender)
MIME-Version: 1.0
Sender: rohityadav89@gmail.com
In-Reply-To: <6DE00C9FDF08A34683DF71786C70EBF02F8CB963@SBPOMB402.sbp.lan>
References: <CD9C1741.182A1%nitin.mehta@citrix.com>
 <F3DB4D3A-C30F-440A-88AB-6A4DAD2A853A@gmail.com>
 <6DE00C9FDF08A34683DF71786C70EBF02F8CB963@SBPOMB402.sbp.lan>
From: Rohit Yadav <bhaisaab@apache.org>
Date: Tue, 23 Apr 2013 23:27:01 +0530
Message-ID: 
 <CANLSSBXWgL=OWq=7YO0338eEuJ3+oqxqAcBk6rLW69D=9DTSpQ@mail.gmail.com>
Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
To: dev@cloudstack.apache.org
Content-Type: multipart/alternative; boundary=089e0160cd76c9a5b104db0aeafd

--089e0160cd76c9a5b104db0aeafd
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Apr 23, 2013 at 5:05 PM, Hugo Trippaers <
HTrippaers@schubergphilis.com> wrote:

> I would favor sticking to the current 4 month release cycle. We have only
> been doing it once and I think we need to have some more experience before
> changing this.
>

+1 Stick with early release if not >4 months release cycle.

I would prefer less than 4 months release cycle, and propose that we "ship
early and ship often". Say 4 months dev overlapping with QA (where say, 2
months dev, 1 months dev+qa feedback/overlap, 1 month bugfixing/qa/feedback
and release).

If we have longer release cycles:

- new stuff would take time to show up
- we would end up breaking a lot of things
- longer feedback and fixing from the real world deployments/prod.
environment

Early releases and a good frequency (i.e. major+minor ones, thanks to jzb
for example we have the 4.0.x hotfix/bugfix/minors) would ensure;

- a good feedback loop and support.
- easier to upgrade, support, have a migration path
- we would be cautious with backward compatibility
- have frequent hotfixes/maintenance releases
- would have new features with better upgrade/paths so when we end up
changing fundamentals stuff such as the architecture, design, structure,
deployment designs it will be less costly and penalizing for both developer
and the user (See 4.1 and 4.2 changelog, woah! the community thought :)

Cheers.


>
> For me another important factor is the amount of features that are coming
> into the system at this moment. With a four month release cycle I would
> have to wait a maximum of four months on the new feature, with 6 months its
> even longer. If we want to get our features out there I think we need to
> release as often as possible so everybody can enjoy the features.  I'd
> rather leave out a few features to ease the burden on QA than delay the
> release.
>
> The same feature thing also has another issue, as David pointed out. The
> longer the development cycle, the more features with hit master. That means
> more testing complexity and an even greater risk when you want to upgrade
> to a new version. And I'm not only talking about our tests, but any user
> will want to test a release before it hits his/her production systems. Make
> the releases to complex and they might be skipped because of testing
> complexity. From a user perspective I'd rather upgrade 3 times a year with
> a smaller change set than two time a year with a bigger changeset. Actually
> I'd rather upgrade 12 times a year with a very small changeset, but that's
> a dream for the future.
>
> Last but not least, if we start delaying releases because we don't have
> the automated test capacity, we will never get it. I believe a bit of
> pressure goes a long way to ensuring we will have automated tests. If
> everybody that is working on features spends just a little bit of time on
> testing, it's done in no time at all. If you look at what happened during
> this release cycle 4.0 to 4.1, I'm really impressed by the level of
> automation that is already there. And much of the groundwork has been
> completed as well so making future test is going to be even easier.
>
> Cheers,
>
> Hugo
>
> > -----Original Message-----
> > From: Sebastien Goasguen [mailto:runseb@gmail.com]
> > Sent: Tuesday, April 23, 2013 9:31 AM
> > To: dev@cloudstack.apache.org
> > Cc: cloudstack-dev@incubator.apache.org; Chip Childers
> > Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
> >
> >
> > On Apr 23, 2013, at 1:32 AM, Nitin Mehta <nitin.mehta@citrix.com> wrote:
> >
> > > David - You have some compelling logic :) Given the debate about 4
> > > month to 6 month, lets make an educated decision
> > >
> > > First - Lets hear from the "release managers" as to what they have to
> > > say about the last 2 releases. What went well and what didn't. Chip - ?
> > > Second - If we plan to do a 6 month release what would be the
> > > timelines in place ? Like time allocated for what features going in
> > > the release, feature discussion and coding, bug fixing, testing,
> > > documentation etc. If I get a rough idea on that for the 6 month
> > > release it would help me understand the differences.
> > >
> > > At this point I would still +1 a 6 month release because it gives me
> > > good time to discuss, code, unit test and bake my feature.
> >
> > We agreed on a time based release cycle, not a feature based one.
> >
> > Moving from 4 to 6 just means that a feature will get delayed 2 months,
> > giving even more time to finish it and test it properly.
> >
> > IMHO, improving quality and reducing QA time comes from increasing
> > testing/review at commit time and making sure feature branches do not
> > diverge from master (too much).
> >
> > I'd rather aim for having releases that have incremental changes and an
> easy
> > upgrade path than a longer release cycle in order to give more time for
> "big"
> > changes that will disrupt upgrade and operations.
> >
> > I also agree with David that we have only gone through the 4 months cycle
> > once (4.1) and this release has some major refactoring. We at least need
> to
> > do it once or twice more to see if there is a real issue.
> >
> > -sebastien
> >
> >
> > > From my experience
> > > at this point this definitely makes more sense.
> > > We can revisit the time line after a couple releases as we all become
> > > more accustomed with the processes in place.
> > >
> > > Thanks,
> > > -Nitin
> > >
> > > On 23/04/13 8:21 AM, "David Nalley" <david@gnsa.us> wrote:
> > >
> > >> On Mon, Apr 22, 2013 at 5:19 PM, Animesh Chaturvedi
> > >> <animesh.chaturvedi@citrix.com> wrote:
> > >>> Folks
> > >>>
> > >>> We started discussing 4 month v/s 6 month release cycle in a another
> > >>> thread [1]. Since the subject of that thread was different,
> > >>> community may not have participated in this important discussion
> > >>> fully. I am  are bringing this discussion to its own thread. Here is
> > >>> the summary so far please refer to [1] for more details.
> > >>>
> > >>> Summary of discussion:
> > >>> - Animesh pointed out the technical debt that we have accumulated so
> > >>> far needs extra time to resolve
> > >>> - David, Chip favor shorter release cycle of 4 month and keeping
> > >>> master always stable and in good quality and enhancing automation as
> > >>> a solution to reduce QA manual effort. A focused defect fixing
> > >>> activity may be needed to reduce technical debt
> > >>> - Will brought up several points in the discussion: He called out
> > >>> heavy dependence on manual QA for a release and pointed out that
> > >>> manual QA may not be always available to match up ACS release
> > >>> schedule. Release overhead for 4 month release is still high and
> > >>> suggest that moving to 6 month will save on release overhead and
> > >>> that  time can be used for strengthening automation.
> > >>> - Joe agrees partly in release overhead being significant for major
> > >>> release
> > >>>
> > >>> If I missed out  any important point please feel free to bring into
> > >>> the thread.
> > >>>
> > >>> There were some other discussion in [1] on release planning
> > >>> conference and chip's clarification on time based v/s feature based
> > >>> releases but we will not discuss those in this thread. Community has
> > >>> agreed to time-based release already.
> > >>>
> > >>> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
> > >>>
> > >>
> > >>
> > >> Hi Animesh:
> > >>
> > >> I thought I would add a few comments. To the folks reading - I
> > >> apologize for the length. If you haven't started yet, you may want to
> > >> get some coffee/tea. You've been warned. :)
> > >>
> > >> I think there are two concerns people think about when talking about
> > >> changing the length of the release cycle.
> > >>
> > >> The first concern is workload. Generating a release has a certain
> > >> fixed amount of overhead. Writing release notes, running votes, and a
> > >> key part of the discussion currently is the QA cycle. Currently a
> > >> large portion of our QA cycle is manual testing, which means we need
> > >> lots of fingers on keyboards to get it done.
> > >>
> > >> The other concern is quality. We always want to try and put our best
> > >> foot forward, and minimize bugs, yet the very act of developing
> > >> software, and especially developing new features, introduces bugs.
> > >>
> > >> Before I start telling you my reasoning, I want to lay out something
> > >> that only hit me tonight. We've talked about how extending the
> > >> release cycle only extends the length of development. We've talked
> > >> about keeping the length of the development period (e.g. pre-code
> > >> freeze) the same, and essentially only extending the amount of time
> for
> > QA.
> > >> That works for the first release after a move from 4 to 6 months. It
> > >> falls apart there after though. The problem is that at code freeze we
> > >> branch. Master immediately becomes open for future feature
> > >> development, and you've just extended it by an additional 2 months,
> > >> because your cycle is that much longer.
> > >>
> > >> I personally don't think that either concern is addressed well by
> > >> lengthening the cycle. Let me explain why
> > >>
> > >> On the quality front - the immediate threat to quality is instability
> > >> inserted by additional development/new features/improvements. By
> > >> increasing the length of time for that disruption to occur, we get
> > >> more of it, as people will continue adding things that might have
> > >> been deferred to a later release. We have code merged that,
> > >> realistically, we won't know if it works properly until QA has gone
> > >> in and verified it, perhaps months later. The more paths changed in a
> > >> given release, the greater the propensity for failure.
> > >
> > > Agreed, but merely changing the time duration doesn't really solve the
> > > problem. More automation, having a staging branch would still do a
> > > better job.
> > > But with the current context what you say makes sense.
> > >
> > >>
> > >> On the overhead front - the overhead of writing release notes and
> > >> generating releases goes down. In addition the 'basic' testing that
> > >> QA does now is additional overhead in each cycle as well. What isn't
> > >> overhead though is testing for new features. Because effectively
> > >> longer cycles mean greater amount of codebase change, more features,
> > >> etc. The actual testing in a given cycle must also increase.
> > >>
> > >> If we were really focused on absolute best quality we could deliver
> > >> with our current tooling, I believe the easiest would be to make our
> > >> release cycle the length of a single feature development period + the
> > >> QA cycle (e.g. shortening the release cycle.)
> > >>
> > >> Is there a right or wrong answer? There shouldn't be. Our software
> > >> should be deliverable at any point in the development cycle we choose.
> > >> 4 months or 6 months would not be a quality, or QA workload issue. We
> > >> would have automated test suites written for 80+% of our testing and
> > >> we would run comprehensive tests against propose merges to guarantee
> > >> that an incoming features both works as intended, and doesn't harm
> > >> the rest of the platform in the process. We'd make release cycle
> > >> decisions based upon the same kind of preferences that make us choose
> > >> a monkey mascot over a polar bear mascot.
> > >>
> > >> That sadly isn't where we are today, but I do think it illuminates a
> > >> clear path of where we need to go. (more automated testing)
> > >>
> > >> So where do I fall on 4 months versus 6 months?
> > >>
> > >> I think we should stay the 4 month course. Heres why:
> > >> * We set the 4-month release cycle expectation already. In theory
> > >> we've been through 2 releases, but really only 4.1 - 4.0 was
> > >> dominated by getting into the incubator, getting releases okayed
> legally,
> > etc.
> > >> It took us 7 months to release. I am loathe to change something we
> > >> only recently set, and have only been through once. I am also worried
> > >> about the signal that such rapid contrary decision changes would send
> > >> to our users.
> > >>
> > >> * I am worried about the impact to quality of our releases. Greater
> > >> introduction of features means an increased threat to stability. It
> > >> may not impact us in the 4.2 cycle, but without massive automated QA
> > >> it will impact us going forward in the 4.3 and later time frames.
> > >>
> > >
> > > Great point and I agree this can definitely be an issue.
> > >
> > >>
> > >> If you've made it thus far, I hope your tea/coffee was enjoyable.
> > >> Thanks for reading. :)
> > >
> > > Had to take 2 :).
> > >
> > >>
> > >> --David
> > >
>
>

--089e0160cd76c9a5b104db0aeafd--