harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: [classlib][test] Migration to testNG?
Date Mon, 16 Jun 2008 10:32:31 GMT
As far as I remember my two years old experiments with TestNG - it
groups all Junit tests (say from all classlib modules) into the one
group while running in JUnit compatibility mode. So the reports
produced by JuntReport didn't look very nice. By the way, I remember
there was some old contribution about xml-based exclude lists for
JUnit that allowed fine-grained (method-based) control over the test
execution. I don't remember exact JIRA number. However, we may search
for it. It was from George Harley.

Regards,
Alexei

2008/6/16, Sian January <sianjanuary@googlemail.com>:
> On 13/06/2008, Sean Qiu <sean.xx.qiu@gmail.com> wrote:
>  >
>  > 2008/6/12 Sian January <sianjanuary@googlemail.com>:
>  > > This is slightly off track, but my concern about TestNG is that there are
>  > > already quite a lot of steps for someone new to Harmony to get the source
>  > > code and run the tests (both for the whole of Harmony and for a single
>  > class
>  > > library module).  Certainly for Eclipse users if we switched to TestNG
>  > this
>  > > would add another step, although I'm not sure about command-line users.
>  > >
>  >
>  > The cost to make use of current tests is low, IMHO.
>  > I have mentioned that we can add TestNG target to replace Junit target.
>  > From the command-line users'  perspective, they still use the same
>  > command to run the test.
>  > The only difference would be the command will invoke TestNG to run our
>  > test.
>  > For Eclipse users, they can adopt plug-in for TestNG, it is quite the
>  > same as plug-in for junit.
>  > So I think we can migration to TestNG smoothly for the user.
>  > What's more, they can pick up their desired tests more conveniently.
>
>
>
> My point was that for Eclipse users it would be an extra step to download
>  the TestNG plug-in, where as the JUnit plugin already comes with Eclipse.  I
>  realise that this isn't the biggest consideration, but I feel that our setup
>  is already quite complex so I thought it would be worth mentioning.  I would
>  assume that switching to JUnit 4 would not require any extra steps because
>  Eclipse already has support for it embedded.
>
>
>  For test developers,  on the other hand, it is a little complex.
>  > But  we want to upgrade to a new tool, no matter Junit 4 or TestNG, we
>  > need to pay for it.Their efforts are similar.
>  > For TestNG, we need add annotations for each testcase, classify them
>  > to right groups.
>  > The good news is we still can use original assert* as before since
>  > TestNG include JUnit's Assert class.
>  > As Nathan mentioned , we can use hamcrest as well if we want.
>  >
>  >
>  >
>  > > Of course if switching to TestNG solves some really major problems then
>  > it
>  > > would probably be worth it, but the only thing I can see that it gives us
>  > > over JUnit 4 is being able to run different sets of tests on different
>  > OS's
>  > > and to me that doesn't seem as important as having a project that's easy
>  > to
>  > > access.  Just my 2p worth...
>  > >
>  >
>  > We can replace the exclude files  to get a more accurate tracking for
>  > failed test.
>  > We can tell difference between public API tests and harmony specific
>  > implementation tests .
>  > We can archive all tests together into a jar.
>  > I think there should be other benefits as well :)
>
>
>
> I think it would be good to be able to reduce the exclude list to just
>  contain individual methods and keep track of it a bit better, although I
>  think some of this can be achieved with JUnit and using different folders.
>  I agree the TestNG solution seems neater and more flexible, I'm just not
>  convinced yet that the benefit we would be getting is worth the cost to
>  upgrade.  I'm also wondering about the overhead for developers.  E.g am I
>  going to have to add four different annotations to every new test I write in
>  the TestNG system?
>
>
>
>  > --
>
> > Best Regards
>  > Sean, Xiao Xia Qiu
>  >
>  > China Software Development Lab, IBM
>  >
>
>
> Thanks,
>
>  Sian
>
>
>  --
>  Unless stated otherwise above:
>  IBM United Kingdom Limited - Registered in England and Wales with number
>  741598.
>  Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Mime
View raw message