incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sudha Ponnaganti <>
Subject RE: [DISCUSS] QA/Testing focus on 4.1
Date Wed, 21 Nov 2012 23:43:43 GMT
Thanks for the report David. Gives good baseline for pre 4.1 
Lot of scope to add unit tests

-----Original Message-----
From: David Nalley [] 
Sent: Wednesday, November 21, 2012 3:39 PM
Subject: Re: [DISCUSS] QA/Testing focus on 4.1

On Wed, Nov 21, 2012 at 6:29 PM, Animesh Chaturvedi <>

> David
> I certainly vote +1  on your ideas to improve testing in CS and 
> enhancing and embracing automation as way of life.  To put in my two 
> cents ,  I would like to highlight  that unit test automation is the 
> base for automation strategy. Here is a good post on test pyramid 
>  As developers if we 
> can commit to comprehensive automated unit tests  we can release early 
> and often. CloudStack with its API layer is already well positioned 
> for service/integration layer automation.
I agree with Martin's sentiment there.
Really what we want to do is get a pipeline going. (and actually now that I write this, I
realize that I need to adjust how things are evolving on
b.a.o) so that it looks like this:
* compile + junit
* docs + apidocs
* package building
* install tests
* smoketests
* full blown integration tests

A failure at any stage should block every other future test. This also gets us from least
expensive (RAT completes in around 1 minute) to most expensive (the full Marvin tests take
hours) Running this frequently also allows us to quickly see where problems have been introduced
because there would be a limited set of changes in each of these test iterations.


View raw message