libcloud-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Carr <johnmdc...@me.com>
Subject [dev] Improving tests
Date Fri, 22 Mar 2013 01:22:23 GMT
Hi all,

A while ago I opened a JIRA ticket for various test improvements I had been thinking about:

https://issues.apache.org/jira/browse/LIBCLOUD-289

It comes with some code, too.


There were a few key points I wanted to address:

 * It is very easy to break things when refactoring. I've managed to slip quite a few bugs
past the tests and review so far, and this is something I want to stop! Anything I can do
to make the tests better at spotting my bugs I have to do.

 * The tests don't enforce our interface. The danger here is that we implement 50 compute
abstractions that sort of work the same. For me, subtle and surprising differences are worse
than using multiple different libraries.

 * The mocks should make more assertions. It's possible to calls a server API but, for example,
not pass any parameters and still have the correct data returned!


For the code referenced in the ticket:

 * There are a shared set of tests that all LB drivers (that I have updated) use. This helps
to pin down the interface and ensure all drivers have the same behaviour.

 * Throws out static fixtures in favour of a mock that "works" — if you call create twice
and then list, you will see 2 records.

 * These mocks are checking much more - first of all that parameters are actually transmitted
to the server, and in the AWS case I do some checks of the auth.


The change in approach uncovered bugs and incompatible interfaces. For example, some implementations
are using strings for ports and some integers. If memory serves (I wrote this at the start
of feb) some use both. Some return values are of different types. 


The code isn't finished yet, but reached a point where I wanted some feedback before continuing.
Initially I was going to just update the load balancers, and then DNS and others as separate
tickets.

Cheers,
John


Mime
View raw message