cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasanna Santhanam <...@apache.org>
Subject Re: [MERGE] Merge VMSync improvement branch into master
Date Fri, 28 Jun 2013 06:16:04 GMT
> > On Jun 27, 2013, at 7:35 PM, David Nalley <david@gnsa.us> wrote:
> >> 
> >> So the problem in my mind, is that we don't have a way of verifying
> >> that master isn't broken, and won't be broken by any given merge. I
> >> look at even the minimal level of automated testing that I see today,
> >> and ~20% of integration tests are failing[1]. The regression set of
> >> tests (which isn't running as often) is seeing 75% of tests
> >> failing[2]. Heaping on more change when we are demonstrably already
> >> failing in many places is not behaving responsibly IMO.
> >> The question I'd pose is this - running the various automated tests is
> >> pretty cheap - whats the output of that compared to the current test
> >> output on master? Better or worse? If it hasn't been done, why not?
> >> I desperately want these features, but not necessarily at the cost of
> >> further destabilizing what we have now in master - we can't continue
> >> accruing technical debt.
> >> 
> >> --David
> >> 
> >> [1] http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-smoke-matrix/lastCompletedBuild/testReport/
> >> [2] http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression-matrix/28/testReport/

*We* need to fix those tests. I wouldn't be thrown off by the numbers
there since many are failing because of mismatched environments and/or
script failures. Getting the same pass rate across the vmsync and
master branches is a _decent_ indicator but not the best one because
coverage is poor. Reviews are absolutely *essential*

In fact it's not easy to automate failures in a standard way for
vmsync like functionality.

-- 
Prasanna.,

------------------------
Powered by BigRock.com


Mime
View raw message