cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Weber <terbol...@gmail.com>
Subject Re: 4.5 RM
Date Tue, 19 Aug 2014 19:33:42 GMT
On Tue, Aug 19, 2014 at 7:15 PM, David Nalley <david@gnsa.us> wrote:

> >
> > IMHO we should not even release 4.5 until we have a agreed upon:
> >
> > -what our issues are and why we released 4.4 and 4.3 late.
> > -taken action to resolve those issues
> > -guarantees that 4.5 will be on time
> >
> > If we don't do that, I don't even know why we are putting ourselves
> through the pain of a release schedule.
> >
>
> So I've been trying to give this some thought. Here's my current line
> of thinking.
>
> The issues with late releases are not a function of our release
> process per se; but are instead a function of our development process.
> CloudStack is a relatively large codebase. It has a lots of points
> that interact with each other, and it's moderately complex.
> Development moves forward and at least happy-path testing is done for
> new features, but the range of options is so large that testing
> everything is a bit difficult. When someone makes a merge request; I
> suspect few people do much looking. Understandable, it's a boring
> task; and really looking doesn't tell us much except for style and
> egregious errors. We've rarely done mandatory testing of feature
> branches before they are merged in. If you want to ship on time, you
> must ensure that we are vociferously guarding the quality of the
> master and release branches; that we can verify programmatically that
> a commit or merge doesn't break things. We must insist on automated
> testing being added.
>
> So I've said all of that to say that I think that ship has sailed for
> 4.5. We are well past feature freeze; and we didn't really have any
> gating functionality. We frankly have very little idea of quality of
> whats in master right now. It's certainly worse than 4.4. So now we'll
> enter code freeze, we'll try and play catch up and fix all of the
> things we discover that are broken. And invariably, we'll be late
> again.
>
> If you want to solve this problem; my personal belief is that its
> really is tied to CI. Efforts around Travis are interesting and
> perhaps are a piece of that puzzle. Discussions around running CI are
> important as well, but I truly believe that we need a gating function
> that prohibits commits that increase our level of untested code or
> code that fails to pass testing. I've seen some other projects using
> pull requests in github, and then using the github pull request
> builder[1] for jenkins to verify that every PR works. I know we've
> talked about gerrit previously, and perhaps that will work as well.
>
> [1] https://wiki.cloudbees.com/bin/view/DEV/Github+Pull+Request+Validation
>


A lot of valid points.

One of my personal beliefs is that we need a better CI solution/system,
rather than depending on the simulator.
There seems to be some sort of consensus/habit of trusting the simulator
output, but both 4.3 and 4.4 has/had some _serious_ issues that the
simulator didn't/doesn't catch.

Sometimes the problem lies in the systemvm, sometimes it could be a quirk
on the hypervisor, it doesn't really matter when you don't actually deploy
the essential parts of an acs installation to test them.

Unfortunately I have no good solution, for that I am far too new to the
whole acs ecosystem

-- 
Erik

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message