harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Liang <richard.lian...@gmail.com>
Subject Re: [testing] metadata approach
Date Wed, 02 Aug 2006 02:17:05 GMT


Alexei Zakharov wrote:
> Hi Richard,
>
>> Not sure if we really want to involve another migration: TestNG javadoc
>> -> TestNG annotation. Any comments?
>
> Well, IMHO this depends on time constraints - when do we plan to have
> the support for anotations? If the answer is about a couple of weeks -
> no problem, we can wait. But if this is several months...
> About the "migration" - I don't think this will be a real painfull
> migration, all infrastructure will remain the same. We will only need
> to convert javadocs to annotations (one-one correspondence) and this
> task can be easily automated.
Sounds reasonable. :-)  Maybe drlvm guys or Oliver could tell us when we 
will have a VM with annotation support?

>
> Thanks,
>
> 2006/8/1, Richard Liang <richard.liangyx@gmail.com>:
>>
>>
>> Alexei Zakharov wrote:
>> > Hi,
>> >
>> > I have created this new thread as a single place for discussions
>> > started in "Re: [testing] Peace" and "[classlib] Testing conventions –
>> > a proposal" threads.
>> >
>> > What did we have in the previous threads?
>> > * Test classification proposed by Vladimir
>> > * Test classification and group names proposed by George
>> > * Solution for Ant and TestNG scripting issues (still being discussed)
>> >
>> > Since a lot of people are asking about TestNG and wanting TestNG I
>> > decide to put some effort and take a closer look at this piece of
>> > software. Thus during the last few days I was playing with TestNG - I
>> > tried to run different kind of tests with it, to perform various
>> > workloads, generate reports in different ways and etc. The purpose of
>> > all this activity was to try TestNG in a real work, understand is
>> > TestNG really worth our credits and how expensive can be moving to
>> > TestNG from our currently implemented testing infrastructure. Now I
>> > have some thoughts and facts I'd like to share with the community.
>> > I've put it in the form of list for convenience.
>> >
>> > * TestNG works ok in normal conditions, no visible bugs
>> > * It is possible to define and use various (possibly intersecting)
>> > test groups with TestNG
>> > * TestNG-style metadata is more convenient than JUnit test suites (now
>> > I agree with this statement). IMHO this is the main TestNG benefit.
>> > * It is possible to run TestNG from command line
>> > * There is also the special ant task for running TestNG
>> > * Not everything can be configured with the ant task or command-line
>> > params, sometimes extra test suite definition file "testng.xml" is
>> > needed
>> > * It is possible to run jUnit tests with TestNG ("testng.xml" is
>> > needed in this case)
>> > * It is possible to run junit tests we currently have in Harmony with
>> > TestNG without any problems and modifications of the source code.
>> > However, we probably should write some number of TestNG test suite
>> > definition files "testng.xml" to be able to run all our junit tests (I
>> > have tried tests for bean module and some tests for luni)
>> > * We can mix jUnit tests and TestNG tests in the single test suite
>> > configuration – i.e. single testng.xml file. We can add TestNG
>> > metadata to some test classes and leave other test classes untouched
>> > * TestNG generates HTML reports in its own style, not a very
>> > convenient one IMHO
>> > * It is also possible to generate JUnitReports from the output
>> > generated by TestNG
>> > * Such reports will have a little bit different structure since TestNG
>> > doesn't provider any information about enclosing type for test
>> > methods. Names for tests (replacement for JUnit "test classes") and
>> > test suites should be externally configured in "testng.xml"
>> > * TestNG for Java 5 doesn't work on Harmony because some necessary
>> > classes from java.util.concurrent package are missing and it seems
>> > that jsr14 target (we are currently using) doesn't support annotations
>> > * TestNG for Java 1.4 (javadoc version) currently works on Harmony
>> > * I have half-way done script that converts TestNG 1.4 metadata
>> > (javadoc) tests to TestNG 1.5 (5.0 annotations) tests.
>> >
>>
>> Excellent summary! Thanks a lot
>>
>> > The question I'd like to raise now is – aren't we ready for TestNG
>> > right now?
>> I suppose we will use Java 5.0 annotations of TestNG, so it seems now we
>> are not ready for TestNG. But we can continue our feasibility study,
>> just like what you have done, to know if TestNG really meets our
>> requirements or if there are any potential problems. Maybe we could list
>> a prerequisite list. e.g,
>> 1) Harmony can fully self-host TestNG with Java5 annotations
>> 2) Test groups are well-defined and agreed in community
>> 3) Guidelines to write TestNG testcases
>> 4) Take one module to run a pilot case
>> ....
>>
>> Please correct me if I'm wrong
>>
>> > For example, we could replace our harness from jUnit to
>> > TestNG and lazily start converting of some failing and platform
>> > dependent tests to javadoc version of TestNG. The rest of the tests
>> > will remain jUnit in fact. And when our VM will be ready to handle
>> > annotations we can convert all our TestNG 1.4 tests to TestNG 1.5. I
>> > understand that this idea may seem to be too early. But anyway we will
>> > need to change things some day since many people are unhappy with the
>> > current testing infrastructure (me for example).
>> Not sure if we really want to involve another migration: TestNG javadoc
>> -> TestNG annotation. Any comments?
>>
>> >
>> > Thought? Suggestions? Opposite opinions?
>> >
>> > With Best Regards,
>> >
>> >
>>
>> -- 
>> Richard Liang
>> China Software Development Lab, IBM
>
>
>

-- 
Richard Liang
China Software Development Lab, IBM 



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message