harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Liang <richard.lian...@gmail.com>
Subject [Testing Convention] Keep tests small and fast
Date Thu, 30 Mar 2006 06:19:03 GMT

I notice that we put all the test code into one big test method (for 
This way we will lose some benefits of junit and even unit test:
1. Test code cannot share configuration code through setUp and tearDown
2. We have to add redundant code, such as, "Assert 1:", "Assert 2: ...." 
to make the test results more comprehensive
3. It makes the test code more complex

Shall we just use small test cases?

You may want to read the description at: 

*Keep tests small and fast*
Executing every test for the entire system shouldn't take hours. Indeed, 
developers will more consistently run tests that execute quickly. 
Without regularly running the full set of tests, it will be difficult to 
validate the entire system when changes are made. Errors will start to 
creep back in, and the benefits of unit testing will be lost. This means 
stress tests and load tests for single classes or small frameworks of 
classes shouldn't be run as part of the unit test suite; they should be 
executed separately.

Thanks a lot.

Richard Liang wrote:
> Dears,
> As I cannot find similar pages about testing convention, I just create 
> one with my rough ideas 
> http://wiki.apache.org/harmony/Testing_Convention, so that we can 
> document our decision timely & clearly.
> Geir Magnusson Jr wrote:
>> Leo Simons wrote:
>>> Gentlemen!
>>> On Mon, Mar 27, 2006 at 11:07:51AM +0200, mr A wrote:
>>>> On Monday 27 March 2006 10:14, mr B wrote:
>>>>> On 3/27/06, mr C wrote:
>>>>> [SNIP]
>>>>>> [SNIP]
>>>>>>> [SNIP]
>>>>>>>> On 1/1/2006, mr D wrote:
>>>>>>>>>>>>>>>>>>> [SNIP]
>>>>> Hmmm... Lemme support [SNIP]
>>>> Now let me support [SNIP].
>>> The ASF front page says
>>>   (...) "The Apache projects are characterized by a collaborative, 
>>> consensus
>>>   based development process, " (...)
>>> That's not just some boilerplate. Consensus is a useful thing.
>>> "How should we organize our tests?" has now been the subject of 
>>> debate for
>>> *months* around here, and every now and then much of the same 
>>> discussion is
>>> rehashed.
>> And we're making progress.  IMO, it really helped my thinking to 
>> distinguish formally between the implementation tests and the spec 
>> tests, because that *completely* helped me come to terms with the 
>> whole o.a.h.test.* issue.
>> I now clearly see where o.a.h.test.*.HashMapTest fits, and where 
>> java.util.HashMapTest fits.
>> I don't think our issues were that obvious before, at least to me.  
>> Now, I see clearly.
>>> I think it would be more productive to look for things to agree on 
>>> (such as,
>>> "we don't know, but we can find out", or "we have different ideas on 
>>> that,
>>> but there's room for both", or "this way of doing things is not the 
>>> best one
>>> but the stuff is still useful so let's thank the guy for his work 
>>> anyway")
>>> than to keep delving deeper and deeper into these kinds of 
>>> disagreements.
>>> Of course, the ASF front page doesn't say that "apache projects are
>>> characterized by a *productive* development process". Its just my 
>>> feeling that
>>> for a system as big as harmony we need to be *very* productive.
>> You don't think we're making progress through these discussions?
>>> Think about it. Is your time better spent convincing lots of other 
>>> people to do
>>> their testing differently, or is it better spent writing better tests?
>> The issue isn't about convincing someone to do it differently, but 
>> understanding the full scope of problems, that we need to embrace 
>> both approaches, because they are apples and oranges, and we need 
>> both apples and oranges.  They aren't exclusionary.
>> geir

Richard Liang
China Software Development Lab, IBM 

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message