harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Beyer" <nbe...@kc.rr.com>
Subject [classlib] TestNG v. JUnit (was: RE: [classlib] Testing conventions - a proposal)
Date Tue, 11 Jul 2006 01:01:22 GMT
Not to add another fire to this topic, but with all things being relative,
so far this topic has been comparison of the TestNG and JUnit v3.8. From
what I understand, the latest JUnit v4.1 provides many of the same
annotation features that TestNG does, as well guaranteed compatibility with
JUnit v3-based tests. 

If we were to compare moving to TestNG with upgrading to JUnit 4.1, would
there still be as much value in the proposition to move to TestNG?

-Nathan

> -----Original Message-----
> From: George Harley [mailto:george.c.harley@googlemail.com]
> Sent: Monday, July 10, 2006 3:57 PM
> To: harmony-dev@incubator.apache.org
> Subject: Re: [classlib] Testing conventions - a proposal
> 
> Alexei Zakharov wrote:
> > Hi George,
> >
> >> For the purposes of this discussion it would be fascinating to find out
> >> why you refer to TestNG as being an "unstable" test harness. What is
> >> that statement based on ?
> >
> > My exact statement was referring to TestNG as "probably unstable"
> > rather than simply "unstable". ;)  This statement was based on posts
> > from Richard Liang about the bug in the TestNG migration tool and on
> > common sense. If the project has such an obvious bug in one place it
> > may probably have other bugs in other places. JUnit is quite famous
> > and widely used toolkit that proved to be stable enough. TestNG is
> > neither famous nor widely used. And IMHO it makes sense to be careful
> > with new exciting tools until we *really* need their innovative
> > functionality.
> >
> 
> Hi Alexei,
> 
> Last I heard, Richard posted saying that there was no bug in the
> migration tool [1]. The command line tool is designed to locate JUnit
> tests under a specified location and add the TestNG annotations to them.
> That's what it does.
> 
> You are right to say that it makes sense to be careful in this matter.
> Nobody wants to do anything that affects Harmony in an adverse way.
> 
> Best regards,
> George
> 
> [1]
> http://mail-archives.apache.org/mod_mbox/incubator-harmony-
> dev/200607.mbox/%3c44B1C084.3020408@gmail.com%3e
> 
> 
> >
> > 2006/7/10, George Harley <george.c.harley@googlemail.com>:
> >> Alexei Zakharov wrote:
> >> >> Actually, there's a very valid benefit for using TestNG markers (=
> >> >> annotations/JavaDoc) for grouping tests; the directory structure is
> a
> >> >> tree, whereas the markers can form any slice of tests, and the sets
> >> >
> >> > Concerning TestNG vs JUnit. I just like to pay your attention on the
> >> > fact what it is possible to achieve the same level of test
> >> > grouping/slicing with JUnit TestSuites. You may define any number of
> >> > intersecting suites - XXXAPIFailingSuite, XXXHYSpecificSuite,
> >> > XXXWinSpecificSuite or whatever. Without necessity of migrating to
> new
> >> > (probably unstable) test harness.
> >> > Just my two cents.
> >> >
> >> >
> >>
> >> Hi Alexei,
> >>
> >> You are quite correct that JUnit test suites are another alternative
> >> here. If I recall correctly, their use was discussed in the very early
> >> days of this project but it came to nothing and we instead went down
> the
> >> route of using exclusion filters in the Ant JUnit task. That approach
> >> does not offer much in the way of fine grain control and relies on us
> >> pushing stuff around the repository. Hence the kicking off of this
> >> thread.
> >>
> >> For the purposes of this discussion it would be fascinating to find out
> >> why you refer to TestNG as being an "unstable" test harness. What is
> >> that statement based on ?
> >>
> >> Best regards,
> >> George
> >>
> >>
> >> > 2006/7/8, Alex Blewitt <alex.blewitt@gmail.com>:
> >> >> On 08/07/06, Geir Magnusson Jr <geir@pobox.com> wrote:
> >> >> >
> >> >> > So while I like the annotations, and expect we can use them
> >> >> effectively,
> >> >> > I have an instinctive skepticism of annotations right now
> >> because in
> >> >> > general (in general in Java), I'm not convinced we've used them
> >> enough
> >> >> > to grok good design patterns.
> >> >>
> >> >> There's really no reason to get hung up on the annotations. TestNG
> >> >> works just as well with JavaDoc source comments; annotations are
> only
> >> >> another means to that end. (They're probably a better one for the
> >> >> future, but it's just an implementation detail.)
> >> >>
> >> >> > Now since I still haven't read the thread fully, I'm jumping to
> >> >> > conclusions, taking it to the extreme, etc etc, but my thinking
in
> >> >> > writing the above is that if we bury everything about our test
> >> >> > 'parameter space' in annotations, some of the visible
> >> organization we
> >> >> > have now w/ on-disk layout becomes invisible, and the readable
> >> >> > "summaries" of aspects of testing that we'd have in an XML
> metadata
> >> >> > document (or whatever) also are hard because you need to scan
the
> >> >> > sources to find all instances of annotation "X".
> >> >>
> >> >> I'm hoping that this would be just as applicable to using JavaDoc
> >> >> variants, and that the problem's not with annotations per se.
> >> >>
> >> >> In either case, both are grokkable with tools -- either
> >> >> annotation-savy readers or a JavaDoc tag processor, and it
> >> wouldn't be
> >> >> hard to configure one of those to periodically scan the codebase to
> >> >> generate reports. Furthermore, as long as the annotation X is well
> >> >> defined, *you* don't have to scan it -- you leave it up to TestNG to
> >> >> figure it out.
> >> >>
> >> >> Actually, there's a very valid benefit for using TestNG markers (=
> >> >> annotations/JavaDoc) for grouping tests; the directory structure is
> a
> >> >> tree, whereas the markers can form any slice of tests, and the sets
> >> >> don't need to be strict subsets (with a tree, everything has to be
a
> >> >> strict subset of its parents). That means that it's possible to
> >> define
> >> >> a marker IO to run all the IO tests, or a marker Win32 to run all
> the
> >> >> Win32 tests, and both of those will contain IO-specific Win32 tests.
> >> >> You can't do that in a tree structure without duplicating content
> >> >> somewhere along the line (e.g. /win/io or /io/win). Neither of these
> >> >> scale well, and every time you add a new dimension, you're doubling
> >> >> the structure of the directory, but merely adding a new marker with
> >> >> TestNG. So if you wanted to have (say) boot classpath tests vs api
> >> >> tests, then you'd ahve to have /api/win/io and /boot/win/io (or
> >> >> various permutations as applicable).
> >> >>
> >> >> Most of the directory-based arguments seem to be along the lines of
> >> >> "/api/win/io is better! No, /win/io/api is better!". Just have an
> >> >> 'api', 'win', 'io' TestNG marker, and then let TestNG figure out
> >> which
> >> >> ones to run. You can then even get specific, and only run the
> Windows
> >> >> IO API tests, if you really want -- but if you don't, you get the
> >> >> benefit of being able to run all IO tests (both API and boot).
> >> >>
> >> >> There doesn't seem to be any benefit to having a strict tree-like
> >> >> structure to the tests when it's possible to have a multi-
> dimensional
> >> >> matrix of all possible combinations that's managed by the tool.
> >> >>
> >> >> Alex.
> >
> >
> 
> 
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message