harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Liang <richard.lian...@gmail.com>
Subject Re: [Testing Convention] Keep tests small and fast
Date Thu, 30 Mar 2006 11:51:18 GMT
Stepan Mishura wrote:
> On 3/30/06, Richard Liang  wrote:
>   
>> Dears,
>>
>> I notice that we put all the test code into one big test method (for
>> example,
>>
>> org.apache.harmony.tests.java.util.jar.test_putLjava_lang_ObjectLjava_lang_Object
>> ).
>> This way we will lose some benefits of junit and even unit test:
>> 1. Test code cannot share configuration code through setUp and tearDown
>> 2. We have to add redundant code, such as, "Assert 1:", "Assert 2: ...."
>> to make the test results more comprehensive
>> 3. It makes the test code more complex
>>
>> Shall we just use small test cases?
>>
>> You may want to read the description at:
>> http://www.javaworld.com/javaworld/jw-12-2000/jw-1221-junit_p.html
>>
>> *Keep tests small and fast*
>> Executing every test for the entire system shouldn't take hours. Indeed,
>> developers will more consistently run tests that execute quickly.
>> Without regularly running the full set of tests, it will be difficult to
>> validate the entire system when changes are made. Errors will start to
>> creep back in, and the benefits of unit testing will be lost. This means
>> stress tests and load tests for single classes or small frameworks of
>> classes shouldn't be run as part of the unit test suite; they should be
>> executed separately.
>>     
>
>
>  Hi Richard,
>
> IMHO, this relates to "stress tests and load tests" only. This means that we
> shouldn't put such kind of tests in a 'regular test suite'. The 'regular
> test suite' is used to verify regressions only. Returning back to a test's
> size, I think it is up to developer - we can only recommend not to test all
> functionality in one test case and split independent parts into a number of
> test case. But IMHO we can not fully avoid creating 'redundant code', such
> as, "Assert 1:", "Assert 2: ....". For example, if there is a constructor
> with several parameters and get-methods to return provided parameters then I
> wouldn't create 3 tests instead of the next one:
>
> public void test_Ctor() {
>     Ctor c = new Ctor(param1, param2, param3);
>
>      assertEquals("Assert 1", param1, c.getParam1());
>     assertEquals("Assert 2", param2, c.getParam3());
>      assertEquals("Assert 3", param3, c.getParam2());
> }
>
>   
Hello Stepan,

Sometimes we do have to use several assert to check the expected 
situation. It's true. However, what I mean is, let's take your 
"test_Ctor" as an example, if we want to design many test cases to 
verify the behavior of the constructor, the number of assert may 
increase dramatically.

Do you need just one test method or 4 methods for the following test?

public void test_Ctor() {
     Ctor c = new Ctor(param1, param2, param3);

     assertEquals("Assert 1", param1, c.getParam1());
     assertEquals("Assert 2", param2, c.getParam3());
     assertEquals("Assert 3", param3, c.getParam2());

     Ctor c2 = new Ctor(null, param2, param3);

     assertNull("Assert 4", c2.getParam1());
     assertEquals("Assert 5", param2, c2.getParam3());
     assertEquals("Assert 6", param3, c3.getParam2());

     Ctor c3 = new Ctor(param1, null, param3);

     assertEquals("Assert 7", param1, c3.getParam1());
     assertNull("Assert 8", c3.getParam3());
     assertEquals("Assert 9", param3, c3.getParam2());

     Ctor c4 = new Ctor(param1, param2, null);

     assertEquals("Assert 10", param1, c4.getParam1());
     assertEquals("Assert 11", param2, c4.getParam3());
     assertNull("Assert 12", c4.getParam2());

}



> Thanks,
> Stepan.
>
>
> Thanks a lot.
>   
>> Richard Liang wrote:
>>     
>>> Dears,
>>>
>>> As I cannot find similar pages about testing convention, I just create
>>> one with my rough ideas
>>> http://wiki.apache.org/harmony/Testing_Convention, so that we can
>>> document our decision timely & clearly.
>>>
>>> Geir Magnusson Jr wrote:
>>>       
>>>> Leo Simons wrote:
>>>>         
>>>>> Gentlemen!
>>>>>
>>>>> On Mon, Mar 27, 2006 at 11:07:51AM +0200, mr A wrote:
>>>>>           
>>>>>> On Monday 27 March 2006 10:14, mr B wrote:
>>>>>>             
>>>>>>> On 3/27/06, mr C wrote:
>>>>>>> [SNIP]
>>>>>>>               
>>>>>>>> [SNIP]
>>>>>>>>                 
>>>>>>>>> [SNIP]
>>>>>>>>>                   
>>>>>>>>>> On 1/1/2006, mr D wrote:
>>>>>>>>>>                     
>>>>>>>>>>>>>>>>>>>>> [SNIP]
>>>>>>>>>>>>>>>>>>>>>     
                                     
>>>>>>> Hmmm... Lemme support [SNIP]
>>>>>>>               
>>>>>> Now let me support [SNIP].
>>>>>>             
>>>>> The ASF front page says
>>>>>
>>>>>   (...) "The Apache projects are characterized by a collaborative,
>>>>> consensus
>>>>>   based development process, " (...)
>>>>>
>>>>> That's not just some boilerplate. Consensus is a useful thing.
>>>>>
>>>>> "How should we organize our tests?" has now been the subject of
>>>>> debate for
>>>>> *months* around here, and every now and then much of the same
>>>>> discussion is
>>>>> rehashed.
>>>>>           
>>>> And we're making progress.  IMO, it really helped my thinking to
>>>> distinguish formally between the implementation tests and the spec
>>>> tests, because that *completely* helped me come to terms with the
>>>> whole o.a.h.test.* issue.
>>>>
>>>> I now clearly see where o.a.h.test.*.HashMapTest fits, and where
>>>> java.util.HashMapTest fits.
>>>>
>>>> I don't think our issues were that obvious before, at least to me.
>>>> Now, I see clearly.
>>>>
>>>>         
>>>>> I think it would be more productive to look for things to agree on
>>>>> (such as,
>>>>> "we don't know, but we can find out", or "we have different ideas on
>>>>> that,
>>>>> but there's room for both", or "this way of doing things is not the
>>>>> best one
>>>>> but the stuff is still useful so let's thank the guy for his work
>>>>> anyway")
>>>>> than to keep delving deeper and deeper into these kinds of
>>>>> disagreements.
>>>>>
>>>>> Of course, the ASF front page doesn't say that "apache projects are
>>>>> characterized by a *productive* development process". Its just my
>>>>> feeling that
>>>>> for a system as big as harmony we need to be *very* productive.
>>>>>           
>>>> You don't think we're making progress through these discussions?
>>>>
>>>>         
>>>>> Think about it. Is your time better spent convincing lots of other
>>>>> people to do
>>>>> their testing differently, or is it better spent writing better tests?
>>>>>           
>>>> The issue isn't about convincing someone to do it differently, but
>>>> understanding the full scope of problems, that we need to embrace
>>>> both approaches, because they are apples and oranges, and we need
>>>> both apples and oranges.  They aren't exclusionary.
>>>>
>>>> geir
>>>>
>>>>         
>>>       
>> --
>> Richard Liang
>> China Software Development Lab, IBM
>>
>>
>>
>>     
>
>
> --
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
>   


-- 
Richard Liang
China Software Development Lab, IBM 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message