harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: Re: [classlib] Testing conventions - a proposal
Date Mon, 10 Jul 2006 14:18:45 GMT
> Actually, there's a very valid benefit for using TestNG markers (=
> annotations/JavaDoc) for grouping tests; the directory structure is a
> tree, whereas the markers can form any slice of tests, and the sets

Concerning TestNG vs JUnit. I just like to pay your attention on the
fact what it is possible to achieve the same level of test
grouping/slicing with JUnit TestSuites. You may define any number of
intersecting suites - XXXAPIFailingSuite, XXXHYSpecificSuite,
XXXWinSpecificSuite or whatever. Without necessity of migrating to new
(probably unstable) test harness.
Just my two cents.


2006/7/8, Alex Blewitt <alex.blewitt@gmail.com>:
> On 08/07/06, Geir Magnusson Jr <geir@pobox.com> wrote:
> >
> > So while I like the annotations, and expect we can use them effectively,
> > I have an instinctive skepticism of annotations right now because in
> > general (in general in Java), I'm not convinced we've used them enough
> > to grok good design patterns.
>
> There's really no reason to get hung up on the annotations. TestNG
> works just as well with JavaDoc source comments; annotations are only
> another means to that end. (They're probably a better one for the
> future, but it's just an implementation detail.)
>
> > Now since I still haven't read the thread fully, I'm jumping to
> > conclusions, taking it to the extreme, etc etc, but my thinking in
> > writing the above is that if we bury everything about our test
> > 'parameter space' in annotations, some of the visible organization we
> > have now w/ on-disk layout becomes invisible, and the readable
> > "summaries" of aspects of testing that we'd have in an XML metadata
> > document (or whatever) also are hard because you need to scan the
> > sources to find all instances of annotation "X".
>
> I'm hoping that this would be just as applicable to using JavaDoc
> variants, and that the problem's not with annotations per se.
>
> In either case, both are grokkable with tools -- either
> annotation-savy readers or a JavaDoc tag processor, and it wouldn't be
> hard to configure one of those to periodically scan the codebase to
> generate reports. Furthermore, as long as the annotation X is well
> defined, *you* don't have to scan it -- you leave it up to TestNG to
> figure it out.
>
> Actually, there's a very valid benefit for using TestNG markers (=
> annotations/JavaDoc) for grouping tests; the directory structure is a
> tree, whereas the markers can form any slice of tests, and the sets
> don't need to be strict subsets (with a tree, everything has to be a
> strict subset of its parents). That means that it's possible to define
> a marker IO to run all the IO tests, or a marker Win32 to run all the
> Win32 tests, and both of those will contain IO-specific Win32 tests.
> You can't do that in a tree structure without duplicating content
> somewhere along the line (e.g. /win/io or /io/win). Neither of these
> scale well, and every time you add a new dimension, you're doubling
> the structure of the directory, but merely adding a new marker with
> TestNG. So if you wanted to have (say) boot classpath tests vs api
> tests, then you'd ahve to have /api/win/io and /boot/win/io (or
> various permutations as applicable).
>
> Most of the directory-based arguments seem to be along the lines of
> "/api/win/io is better! No, /win/io/api is better!". Just have an
> 'api', 'win', 'io' TestNG marker, and then let TestNG figure out which
> ones to run. You can then even get specific, and only run the Windows
> IO API tests, if you really want -- but if you don't, you get the
> benefit of being able to run all IO tests (both API and boot).
>
> There doesn't seem to be any benefit to having a strict tree-like
> structure to the tests when it's possible to have a multi-dimensional
> matrix of all possible combinations that's managed by the tool.
>
> Alex.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


-- 
Alexei Zakharov,
Intel Middleware Product Division

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message