Mailing-List: contact dev-help@libcloud.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@libcloud.apache.org
Received-SPF: pass (nike.apache.org: domain of johnmdcarr@me.com designates
 17.172.204.237 as permitted sender)
From: John Carr <johnmdcarr@me.com>
Content-type: text/plain; charset=windows-1252
Content-transfer-encoding: quoted-printable
Message-id: <0FE0897F-2531-418F-915E-6DBAEF472A7A@me.com>
Date: Fri, 22 Mar 2013 01:22:23 +0000
To: dev <dev@libcloud.apache.org>
MIME-version: 1.0 (Mac OS X Mail 6.2 \(1499\))
Subject: [dev] Improving tests

Hi all,

A while ago I opened a JIRA ticket for various test improvements I had =
been thinking about:

https://issues.apache.org/jira/browse/LIBCLOUD-289

It comes with some code, too.


There were a few key points I wanted to address:

 * It is very easy to break things when refactoring. I've managed to =
slip quite a few bugs past the tests and review so far, and this is =
something I want to stop! Anything I can do to make the tests better at =
spotting my bugs I have to do.

 * The tests don't enforce our interface. The danger here is that we =
implement 50 compute abstractions that sort of work the same. For me, =
subtle and surprising differences are worse than using multiple =
different libraries.

 * The mocks should make more assertions. It's possible to calls a =
server API but, for example, not pass any parameters and still have the =
correct data returned!


For the code referenced in the ticket:

 * There are a shared set of tests that all LB drivers (that I have =
updated) use. This helps to pin down the interface and ensure all =
drivers have the same behaviour.

 * Throws out static fixtures in favour of a mock that "works" =97 if =
you call create twice and then list, you will see 2 records.

 * These mocks are checking much more - first of all that parameters are =
actually transmitted to the server, and in the AWS case I do some checks =
of the auth.


The change in approach uncovered bugs and incompatible interfaces. For =
example, some implementations are using strings for ports and some =
integers. If memory serves (I wrote this at the start of feb) some use =
both. Some return values are of different types.=20


The code isn't finished yet, but reached a point where I wanted some =
feedback before continuing. Initially I was going to just update the =
load balancers, and then DNS and others as separate tickets.

Cheers,
John