harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Gray <chris.g...@kiffer.be>
Subject Re: Unit testing revisited
Date Mon, 27 Mar 2006 09:07:51 GMT
On Monday 27 March 2006 10:14, Anton Avtamonov wrote:
> On 3/27/06, Geir Magnusson Jr <geir@pobox.com> wrote:
> [SNIP]
>
> > I don't think it's easier to write test cases for internals though a
> > public API, because you can call the internal methods directly to see if
> > they do as they promise, where it's not so easy to always have the same
> > thing happen through an indirect public API.
> >
> > Also, wrt things breaking..... yes, tests that ensure that something
> > works as expected will need to change if the expectation changes.
> >
> > That also is exactly what you want, right?  To know if you broke a
> > 'internal contract' in a class?
>
> Hmmm... Lemme support Richard's point.

Now let me support Geir's.

> At the first, internals not just specify 'internal functional
> contracts', but design decisions. Making redesign you will probably
> have some (design-dependent) unit tests failing, despite functionality
> is not changed. 

In a complex piece of software it is extremely valuable to have internal 
interfaces which are rather stable - for example the VMI or equivalent, 
interfaces with the thread management system, memory management/GC, JIT 
compiler where present, etc.. These do indeed have the nature of "internal 
functional contracts", because each subgroup of developers trusts the other 
groups not to change these interfaces without warning. The contract can be 
changed without reference to external parties, but it is still a contract. If 
you change the implementation of some component and do not intend to change 
the contract, it is extremely useful to have tests which verify that this is 
so.

For the Wonka VM we had such tests for the internal threading API and for the 
abstract data types (hashtable, fifo, etc.) which were used intensively, and 
they proved extremely valuable. They are especially valuable if you take the 
"first make it correct, then make it time-/space-efficient" approach, so that 
one group can work on tweaking a particular component without disturbing the 
others. We completely re-implemented Wonka's string pool a couple of times 
without having to rewrite the rest of the VM.

> At the second, what we really need to satisfy is API behavior only.
> Internal implementation is not significantly important if public part
> works fine. I fond of XP and test-driven development and don't feel
> that we need to test something which is not part of requirements :-).
> [...]

I'm not fond of initials, but I am fond of testing stuff at multiple levels - 
each module or component must do its thing, and together they must satisfy 
the requirements. If you build a complex system out of untested components 
and test only the final result, disaster beckons.

> Sometimes, it is really desire to write an 'internal' test. For
> instance, if you use some sorting algoritms, or algoritms which find
> next match, etc. In this case such 'utility' stuff might be tested
> independently from the points of usage. However I believe that is not
> 'general case'.

It may not be general, but there are enough specific instances to make it 
worthwhile having a systematic way to do such tests.

Note that I am not advocating full white-box testing, in which you check the 
state of every internal variable after every stimulus - that really is a 
waste of time. I'm advocating black-box testing of internal interfaces which 
are important enough to be worth documenting.

Cheers,

Chris

-- 
Chris Gray        /k/ Embedded Java Solutions      BE0503765045
Embedded & Mobile Java, OSGi    http://www.k-embedded-java.com/
chris.gray@kiffer.be                             +32 3 216 0369


Mime
View raw message