cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastien Goasguen <>
Subject Re: unit tests for api/src/com/cloud/agent/api
Date Tue, 18 Sep 2012 19:23:22 GMT

On Sep 18, 2012, at 6:53 PM, prasanna <> wrote:

> On 18 September 2012 18:20, David Nalley <> wrote:
>> On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <> wrote:
>>> First, thanks for working on test stuff, we really need more automate test cases.
>>> If you plan to work on the API test, we have a python test framework:
The most compelling feature is that it supports code completion for cloudstack API, very easy
to write test case in Eclipse environment. And we are setting up a test infrastructure, which
will execute all the test cases written for this framework.
>>> We already have around 500+ test cases, or maybe more, but apparently it's not
enough. If you like to work on it, that will be great!
>> While I like the functional/integration tests, as you note we have
>> 500+ currently. Those tests also take many hours to run at present.
>> Unit tests should be something fast, a sanity check, and we only have
>> 25 right now for the entire code base, so while I like
>> func/integration tests, I'd really like for our unit tests to be
>> runnable on every commit, and provide some decent level of coverage so
>> that we have a bit of confidence that code works well, and provide
>> good feedback to folks on whether their commit passes the sniff test.
>> --David
> We can speed up the tests. Just never got to have marvin drive
> everything asynchronously.It's been on my TODO list forever now as
> have the API tests. I for one would be extremely happy to see some.

I am gonna look into the speed up. Where you thinking of threading it or using nose which
seems to support multiprocess out of the box ?

> About the tool though, here's my opinion:
> An API test would be a great addition to the marvin suite. Marvin
> makes it easier to write them too. All you will need is compare a
> canned response from say listVirtualMachines with what is returned by
> the CS endpoint. Writing these as python's doctests would make it
> cleaner and easier to maintain too. Having one tool do all functional
> and sanity tests is a bonus as well.
> If you do choose to go the jUnit/java way I suggest you look at
> jClouds as well. It has a cloudstack-ec2 suite that does API tests
> already. They will need to be extended and brought to match with the
> changes done for 4.0.
> Yichi - I think if you begin testing with all the listXxx API calls
> and put them down as unit-tests the createXxx, addXxx, deleteXxx will
> fall in place automatically.

View raw message