harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: [testing] metadata approach
Date Tue, 01 Aug 2006 11:30:08 GMT
Hi Richard,

> Not sure if we really want to involve another migration: TestNG javadoc
> -> TestNG annotation. Any comments?

Well, IMHO this depends on time constraints - when do we plan to have
the support for anotations? If the answer is about a couple of weeks -
no problem, we can wait. But if this is several months...
About the "migration" - I don't think this will be a real painfull
migration, all infrastructure will remain the same. We will only need
to convert javadocs to annotations (one-one correspondence) and this
task can be easily automated.

Thanks,

2006/8/1, Richard Liang <richard.liangyx@gmail.com>:
>
>
> Alexei Zakharov wrote:
> > Hi,
> >
> > I have created this new thread as a single place for discussions
> > started in "Re: [testing] Peace" and "[classlib] Testing conventions –
> > a proposal" threads.
> >
> > What did we have in the previous threads?
> > * Test classification proposed by Vladimir
> > * Test classification and group names proposed by George
> > * Solution for Ant and TestNG scripting issues (still being discussed)
> >
> > Since a lot of people are asking about TestNG and wanting TestNG I
> > decide to put some effort and take a closer look at this piece of
> > software. Thus during the last few days I was playing with TestNG - I
> > tried to run different kind of tests with it, to perform various
> > workloads, generate reports in different ways and etc. The purpose of
> > all this activity was to try TestNG in a real work, understand is
> > TestNG really worth our credits and how expensive can be moving to
> > TestNG from our currently implemented testing infrastructure. Now I
> > have some thoughts and facts I'd like to share with the community.
> > I've put it in the form of list for convenience.
> >
> > * TestNG works ok in normal conditions, no visible bugs
> > * It is possible to define and use various (possibly intersecting)
> > test groups with TestNG
> > * TestNG-style metadata is more convenient than JUnit test suites (now
> > I agree with this statement). IMHO this is the main TestNG benefit.
> > * It is possible to run TestNG from command line
> > * There is also the special ant task for running TestNG
> > * Not everything can be configured with the ant task or command-line
> > params, sometimes extra test suite definition file "testng.xml" is
> > needed
> > * It is possible to run jUnit tests with TestNG ("testng.xml" is
> > needed in this case)
> > * It is possible to run junit tests we currently have in Harmony with
> > TestNG without any problems and modifications of the source code.
> > However, we probably should write some number of TestNG test suite
> > definition files "testng.xml" to be able to run all our junit tests (I
> > have tried tests for bean module and some tests for luni)
> > * We can mix jUnit tests and TestNG tests in the single test suite
> > configuration – i.e. single testng.xml file. We can add TestNG
> > metadata to some test classes and leave other test classes untouched
> > * TestNG generates HTML reports in its own style, not a very
> > convenient one IMHO
> > * It is also possible to generate JUnitReports from the output
> > generated by TestNG
> > * Such reports will have a little bit different structure since TestNG
> > doesn't provider any information about enclosing type for test
> > methods. Names for tests (replacement for JUnit "test classes") and
> > test suites should be externally configured in "testng.xml"
> > * TestNG for Java 5 doesn't work on Harmony because some necessary
> > classes from java.util.concurrent package are missing and it seems
> > that jsr14 target (we are currently using) doesn't support annotations
> > * TestNG for Java 1.4 (javadoc version) currently works on Harmony
> > * I have half-way done script that converts TestNG 1.4 metadata
> > (javadoc) tests to TestNG 1.5 (5.0 annotations) tests.
> >
>
> Excellent summary! Thanks a lot
>
> > The question I'd like to raise now is – aren't we ready for TestNG
> > right now?
> I suppose we will use Java 5.0 annotations of TestNG, so it seems now we
> are not ready for TestNG. But we can continue our feasibility study,
> just like what you have done, to know if TestNG really meets our
> requirements or if there are any potential problems. Maybe we could list
> a prerequisite list. e.g,
> 1) Harmony can fully self-host TestNG with Java5 annotations
> 2) Test groups are well-defined and agreed in community
> 3) Guidelines to write TestNG testcases
> 4) Take one module to run a pilot case
> ....
>
> Please correct me if I'm wrong
>
> > For example, we could replace our harness from jUnit to
> > TestNG and lazily start converting of some failing and platform
> > dependent tests to javadoc version of TestNG. The rest of the tests
> > will remain jUnit in fact. And when our VM will be ready to handle
> > annotations we can convert all our TestNG 1.4 tests to TestNG 1.5. I
> > understand that this idea may seem to be too early. But anyway we will
> > need to change things some day since many people are unhappy with the
> > current testing infrastructure (me for example).
> Not sure if we really want to involve another migration: TestNG javadoc
> -> TestNG annotation. Any comments?
>
> >
> > Thought? Suggestions? Opposite opinions?
> >
> > With Best Regards,
> >
> >
>
> --
> Richard Liang
> China Software Development Lab, IBM



-- 
Alexei Zakharov,
Intel Middleware Product Division

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message