incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sudha Ponnaganti <sudha.ponnaga...@citrix.com>
Subject RE: [DISCUSS] QA/Testing focus on 4.1
Date Wed, 21 Nov 2012 23:43:43 GMT
Thanks for the report David. Gives good baseline for pre 4.1 
Lot of scope to add unit tests 

https://builds.apache.org/job/cloudstack-master-maven/13/cobertura/



-----Original Message-----
From: David Nalley [mailto:david@gnsa.us] 
Sent: Wednesday, November 21, 2012 3:39 PM
To: cloudstack-dev@incubator.apache.org
Subject: Re: [DISCUSS] QA/Testing focus on 4.1

On Wed, Nov 21, 2012 at 6:29 PM, Animesh Chaturvedi < animesh.chaturvedi@citrix.com>
wrote:

> David
>
> I certainly vote +1  on your ideas to improve testing in CS and 
> enhancing and embracing automation as way of life.  To put in my two 
> cents ,  I would like to highlight  that unit test automation is the 
> base for automation strategy. Here is a good post on test pyramid 
> http://martinfowler.com/bliki/TestPyramid.html.  As developers if we 
> can commit to comprehensive automated unit tests  we can release early 
> and often. CloudStack with its API layer is already well positioned 
> for service/integration layer automation.
>
>
I agree with Martin's sentiment there.
Really what we want to do is get a pipeline going. (and actually now that I write this, I
realize that I need to adjust how things are evolving on
b.a.o) so that it looks like this:
* RAT
* compile + junit
* docs + apidocs
* package building
* install tests
* smoketests
* full blown integration tests

A failure at any stage should block every other future test. This also gets us from least
expensive (RAT completes in around 1 minute) to most expensive (the full Marvin tests take
hours) Running this frequently also allows us to quickly see where problems have been introduced
because there would be a limited set of changes in each of these test iterations.

--David

Mime
View raw message