harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geir Magnusson Jr <g...@pobox.com>
Subject Re: Unit testing revisited
Date Mon, 27 Mar 2006 09:38:18 GMT
Thx Chris - this expresses very well how I look at it.


Chris Gray wrote:
> On Monday 27 March 2006 10:14, Anton Avtamonov wrote:
>> On 3/27/06, Geir Magnusson Jr <geir@pobox.com> wrote:
>> [SNIP]
>>> I don't think it's easier to write test cases for internals though a
>>> public API, because you can call the internal methods directly to see if
>>> they do as they promise, where it's not so easy to always have the same
>>> thing happen through an indirect public API.
>>> Also, wrt things breaking..... yes, tests that ensure that something
>>> works as expected will need to change if the expectation changes.
>>> That also is exactly what you want, right?  To know if you broke a
>>> 'internal contract' in a class?
>> Hmmm... Lemme support Richard's point.
> Now let me support Geir's.
>> At the first, internals not just specify 'internal functional
>> contracts', but design decisions. Making redesign you will probably
>> have some (design-dependent) unit tests failing, despite functionality
>> is not changed. 
> In a complex piece of software it is extremely valuable to have internal 
> interfaces which are rather stable - for example the VMI or equivalent, 
> interfaces with the thread management system, memory management/GC, JIT 
> compiler where present, etc.. These do indeed have the nature of "internal 
> functional contracts", because each subgroup of developers trusts the other 
> groups not to change these interfaces without warning. The contract can be 
> changed without reference to external parties, but it is still a contract. If 
> you change the implementation of some component and do not intend to change 
> the contract, it is extremely useful to have tests which verify that this is 
> so.
> For the Wonka VM we had such tests for the internal threading API and for the 
> abstract data types (hashtable, fifo, etc.) which were used intensively, and 
> they proved extremely valuable. They are especially valuable if you take the 
> "first make it correct, then make it time-/space-efficient" approach, so that 
> one group can work on tweaking a particular component without disturbing the 
> others. We completely re-implemented Wonka's string pool a couple of times 
> without having to rewrite the rest of the VM.
>> At the second, what we really need to satisfy is API behavior only.
>> Internal implementation is not significantly important if public part
>> works fine. I fond of XP and test-driven development and don't feel
>> that we need to test something which is not part of requirements :-).
>> [...]
> I'm not fond of initials, but I am fond of testing stuff at multiple levels - 
> each module or component must do its thing, and together they must satisfy 
> the requirements. If you build a complex system out of untested components 
> and test only the final result, disaster beckons.
>> Sometimes, it is really desire to write an 'internal' test. For
>> instance, if you use some sorting algoritms, or algoritms which find
>> next match, etc. In this case such 'utility' stuff might be tested
>> independently from the points of usage. However I believe that is not
>> 'general case'.
> It may not be general, but there are enough specific instances to make it 
> worthwhile having a systematic way to do such tests.
> Note that I am not advocating full white-box testing, in which you check the 
> state of every internal variable after every stimulus - that really is a 
> waste of time. I'm advocating black-box testing of internal interfaces which 
> are important enough to be worth documenting.
> Cheers,
> Chris

View raw message