harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stepan Mishura" <stepan.mish...@gmail.com>
Subject Re: [Testing Convention] Keep tests small and fast
Date Thu, 30 Mar 2006 07:44:02 GMT
On 3/30/06, Richard Liang  wrote:
>
> Dears,
>
> I notice that we put all the test code into one big test method (for
> example,
>
> org.apache.harmony.tests.java.util.jar.test_putLjava_lang_ObjectLjava_lang_Object
> ).
> This way we will lose some benefits of junit and even unit test:
> 1. Test code cannot share configuration code through setUp and tearDown
> 2. We have to add redundant code, such as, "Assert 1:", "Assert 2: ...."
> to make the test results more comprehensive
> 3. It makes the test code more complex
>
> Shall we just use small test cases?
>
> You may want to read the description at:
> http://www.javaworld.com/javaworld/jw-12-2000/jw-1221-junit_p.html
>
> *Keep tests small and fast*
> Executing every test for the entire system shouldn't take hours. Indeed,
> developers will more consistently run tests that execute quickly.
> Without regularly running the full set of tests, it will be difficult to
> validate the entire system when changes are made. Errors will start to
> creep back in, and the benefits of unit testing will be lost. This means
> stress tests and load tests for single classes or small frameworks of
> classes shouldn't be run as part of the unit test suite; they should be
> executed separately.


 Hi Richard,

IMHO, this relates to "stress tests and load tests" only. This means that we
shouldn't put such kind of tests in a 'regular test suite'. The 'regular
test suite' is used to verify regressions only. Returning back to a test's
size, I think it is up to developer - we can only recommend not to test all
functionality in one test case and split independent parts into a number of
test case. But IMHO we can not fully avoid creating 'redundant code', such
as, "Assert 1:", "Assert 2: ....". For example, if there is a constructor
with several parameters and get-methods to return provided parameters then I
wouldn't create 3 tests instead of the next one:

public void test_Ctor() {
    Ctor c = new Ctor(param1, param2, param3);

     assertEquals("Assert 1", param1, c.getParam1());
    assertEquals("Assert 2", param2, c.getParam3());
     assertEquals("Assert 3", param3, c.getParam2());
}

Thanks,
Stepan.


Thanks a lot.
>
> Richard Liang wrote:
> > Dears,
> >
> > As I cannot find similar pages about testing convention, I just create
> > one with my rough ideas
> > http://wiki.apache.org/harmony/Testing_Convention, so that we can
> > document our decision timely & clearly.
> >
> > Geir Magnusson Jr wrote:
> >>
> >>
> >> Leo Simons wrote:
> >>> Gentlemen!
> >>>
> >>> On Mon, Mar 27, 2006 at 11:07:51AM +0200, mr A wrote:
> >>>> On Monday 27 March 2006 10:14, mr B wrote:
> >>>>> On 3/27/06, mr C wrote:
> >>>>> [SNIP]
> >>>>>> [SNIP]
> >>>>>>> [SNIP]
> >>>>>>>> On 1/1/2006, mr D wrote:
> >>>>>>>>>>>>>>>>>>> [SNIP]
> >>>>> Hmmm... Lemme support [SNIP]
> >>>> Now let me support [SNIP].
> >>>
> >>> The ASF front page says
> >>>
> >>>   (...) "The Apache projects are characterized by a collaborative,
> >>> consensus
> >>>   based development process, " (...)
> >>>
> >>> That's not just some boilerplate. Consensus is a useful thing.
> >>>
> >>> "How should we organize our tests?" has now been the subject of
> >>> debate for
> >>> *months* around here, and every now and then much of the same
> >>> discussion is
> >>> rehashed.
> >>
> >> And we're making progress.  IMO, it really helped my thinking to
> >> distinguish formally between the implementation tests and the spec
> >> tests, because that *completely* helped me come to terms with the
> >> whole o.a.h.test.* issue.
> >>
> >> I now clearly see where o.a.h.test.*.HashMapTest fits, and where
> >> java.util.HashMapTest fits.
> >>
> >> I don't think our issues were that obvious before, at least to me.
> >> Now, I see clearly.
> >>
> >>>
> >>> I think it would be more productive to look for things to agree on
> >>> (such as,
> >>> "we don't know, but we can find out", or "we have different ideas on
> >>> that,
> >>> but there's room for both", or "this way of doing things is not the
> >>> best one
> >>> but the stuff is still useful so let's thank the guy for his work
> >>> anyway")
> >>> than to keep delving deeper and deeper into these kinds of
> >>> disagreements.
> >>>
> >>> Of course, the ASF front page doesn't say that "apache projects are
> >>> characterized by a *productive* development process". Its just my
> >>> feeling that
> >>> for a system as big as harmony we need to be *very* productive.
> >>
> >> You don't think we're making progress through these discussions?
> >>
> >>>
> >>> Think about it. Is your time better spent convincing lots of other
> >>> people to do
> >>> their testing differently, or is it better spent writing better tests?
> >>
> >> The issue isn't about convincing someone to do it differently, but
> >> understanding the full scope of problems, that we need to embrace
> >> both approaches, because they are apples and oranges, and we need
> >> both apples and oranges.  They aren't exclusionary.
> >>
> >> geir
> >>
> >
> >
>
>
> --
> Richard Liang
> China Software Development Lab, IBM
>
>
>


--
Thanks,
Stepan Mishura
Intel Middleware Products Division

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message