river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher Dolan" <christopher.do...@avid.com>
Subject RE: test framework migration - was: Re: Hudson build is back to normal
Date Fri, 12 Nov 2010 18:03:49 GMT
Last year my project switched all of our tests from Junit 3 to TestNG.  My big win with TestNG
vs. Junit 4 was the ability to classify the tests and decide at runtime which to run.  Most
of my tests are annotated with @Test(groups="build") which are the tests executed by my continuous
integration process.  Only a subset of my code is Mac-specific, so those tests are marked
as @Test(groups={"build","macos"}).  My really slow tests are marked as @Test(groups="all")
and are only executed on specific machines.

>From Ant, I choose which groups to run via <testng groups="build" ... /> or <testng
groups="macos" ... /> etc.

TestNG also has an @DataProvider notation that dramatically simplifies groups of tests that
have the exact same code but different initial conditions.  Here's a too-simple example just
to show the technique:

   public Object[][] getTestcases() {
        return new Object[][] {
            { 1, "1" },
            { -0, "0" },

   @Test(groups="build", dataProvider="testcases")
   public void testNumification(int num, String expected) {
       Assert.assertEquals("" + num, expected);

Another useful annotation regards exceptions, which says to fail the test unless an exception
class (or subclass) is thrown

   @Test(groups="build", expectedException=IllegalArgumentException.class)
   public void testParseIntFailure() {
       Integer.parseInt("not a number");

TestNG also lets you throw org.testng.SkipException at runtime to neither pass nor fail the
test.  For example, if I want to run a test on Win7 but skip it on WinXP.

But, I agree with Peter that it has to be all TestNG or all JUnit.  You can make them play
nice, but it's not worth the hassle.  The conversion was a little tedious (do NOT trust the
auto-convert tools!  Convert by hand instead!) but bearable.  TestNG has their own org.testng.Assert
that has a different API from org.junit.Assert, but both throw AssertionError in the end so
it doesn't actually matter which you use. I like TestNG's API a little better (methods arguments
are ordered "got, expected, message" vs. Junit's "message, expected, got").


-----Original Message-----
From: Peter Firmstone [mailto:jini@zeus.net.au] 
Sent: Friday, November 12, 2010 4:06 AM
To: river-dev@incubator.apache.org
Subject: Re: test framework migration - was: Re: Hudson build is back to normal

Thanks Tom, I agree, if someone has a good reason...



Tom Hobbs wrote:
> I've written a couple of JUnit tests.
> I've not used TestNG for many years so can't really comment on it's
> comparison with JUnit.  I don't object to converting my tests, however
> I'd rather convert them because "TestNG does X which we need to do Y"
> rather than "We should swap to TestNG because it's better."
> I agree, this should be an either/or decision.  We should definitely
> *not* be using both.
> On Fri, Nov 12, 2010 at 1:04 AM, Peter Firmstone <jini@zeus.net.au> wrote:
>> Patricia Shanahan wrote:
>>> Zsolt Kúti wrote:
>>>> On Sat, 6 Nov 2010 17:40:19 +0100
>>>> Jonathan Costers <jonathan.costers@googlemail.com> wrote:
>>>> ...
>>>>> Not all of them use/need a multi VM setup. Those are candidates for
>>>>> JUnit. The others would be QA candidates.
>>>>> I'm not saying it is easy to migrate any of these though, doing so
>>>>> requires knowledge of how the jtreg framework operates, as well as
>>>>> the proposed target framework (JUnit, QA).
>>>>>> JUnit's good when we're only testing a single object
>>>>>> implementation, we can document and expect people to utilse the qa
>>>>>> suite for more complex tests.
>>>>> Agreed.
>>>> Hello hard workers,
>>>> It would be worth considering the use of TestNG instead of JUnit.
>>>> I have no experience in their comparison, so relied on other
>>>> sources when I was to decid what framework to use (like this:
>>>> http://www.mkyong.com/unittest/junit-4-vs-testng-comparison/).
>>>> TestNG features that are missing from JUnit can be useful in a complex
>>>> test environment like that of River.
>>> If we were starting cold, with no existing tests, I might be open to this
>>> suggestion. As it is, we already have a QA framework that can do all the
>>> complex, multi-JVM tests, and we have over 1000 existing tests using it.
>>> I think the objective in converting jtreg tests would be to reduce the
>>> number of frameworks, and the amount of software we need installed, in order
>>> to run a full test. Switching them to TestNG, or anything else other than
>>> JUnit or the River QA framework, would not achieve that.
>>> Patricia
>> Has anyone else written any junit tests other than myself? If TestNG is
>> justifiably better than Junit, I'd be prepared to convert my tests, I
>> believe many of the annotations are common?
>> But it would have to be Junit OR TestNG, not both.
>> Cheers,
>> Peter.

View raw message