From harmony-dev-return-11257-apmail-incubator-harmony-dev-archive=incubator.apache.org@incubator.apache.org Wed Aug 02 02:20:20 2006 Return-Path: Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org Received: (qmail 68566 invoked from network); 2 Aug 2006 02:20:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 2 Aug 2006 02:20:20 -0000 Received: (qmail 58493 invoked by uid 500); 2 Aug 2006 02:20:13 -0000 Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org Received: (qmail 58446 invoked by uid 500); 2 Aug 2006 02:20:13 -0000 Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: harmony-dev@incubator.apache.org Delivered-To: mailing list harmony-dev@incubator.apache.org Received: (qmail 58434 invoked by uid 99); 2 Aug 2006 02:20:13 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Aug 2006 19:20:13 -0700 X-ASF-Spam-Status: No, hits=1.9 required=10.0 tests=DNS_FROM_RFC_ABUSE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (asf.osuosl.org: 202.81.18.154 is neither permitted nor denied by domain of richard.liangyx@gmail.com) Received: from [202.81.18.154] (HELO ausmtp05.au.ibm.com) (202.81.18.154) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Aug 2006 19:20:12 -0700 Received: from sd0208e0.au.ibm.com (d23rh904.au.ibm.com [202.81.18.202]) by ausmtp05.au.ibm.com (8.13.6/8.13.6) with ESMTP id k722MU3U8003760 for ; Wed, 2 Aug 2006 12:22:30 +1000 Received: from d23av02.au.ibm.com (d23av02.au.ibm.com [9.190.250.243]) by sd0208e0.au.ibm.com (8.13.6/8.13.6/NCO v8.1) with ESMTP id k722N6Mq244230 for ; Wed, 2 Aug 2006 12:23:09 +1000 Received: from d23av02.au.ibm.com (loopback [127.0.0.1]) by d23av02.au.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id k722Hhtg024405 for ; Wed, 2 Aug 2006 12:17:43 +1000 Received: from d23m0011.cn.ibm.com ([9.181.32.74]) by d23av02.au.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id k722HJkj021844 for ; Wed, 2 Aug 2006 12:17:42 +1000 Received: from [127.0.0.1] ([9.181.106.60]) by d23m0011.cn.ibm.com (Lotus Domino Release 6.5.5HF262) with ESMTP id 2006080210190269-18471 ; Wed, 2 Aug 2006 10:19:02 +0800 Message-ID: <44D00B21.3030807@gmail.com> Date: Wed, 02 Aug 2006 10:17:05 +0800 From: Richard Liang User-Agent: Thunderbird 1.5.0.5 (Windows/20060719) MIME-Version: 1.0 To: harmony-dev@incubator.apache.org Subject: Re: [testing] metadata approach References: <2c9597b90607280631p2b4f6fefldaf4ff1c5cd00406@mail.gmail.com> <44CF0FEB.5050404@gmail.com> <2c9597b90608010430r5424c270mcbd19cdcb1570ae5@mail.gmail.com> In-Reply-To: <2c9597b90608010430r5424c270mcbd19cdcb1570ae5@mail.gmail.com> X-MIMETrack: Itemize by SMTP Server on D23M0011/23/M/IBM(Release 6.5.5HF262 | April 5, 2006) at 02/08/2006 10:19:02, Serialize by Router on D23M0011/23/M/IBM(Release 6.5.5HF262 | April 5, 2006) at 02/08/2006 10:19:40 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Alexei Zakharov wrote: > Hi Richard, > >> Not sure if we really want to involve another migration: TestNG javadoc >> -> TestNG annotation. Any comments? > > Well, IMHO this depends on time constraints - when do we plan to have > the support for anotations? If the answer is about a couple of weeks - > no problem, we can wait. But if this is several months... > About the "migration" - I don't think this will be a real painfull > migration, all infrastructure will remain the same. We will only need > to convert javadocs to annotations (one-one correspondence) and this > task can be easily automated. Sounds reasonable. :-) Maybe drlvm guys or Oliver could tell us when we=20 will have a VM with annotation support? > > Thanks, > > 2006/8/1, Richard Liang : >> >> >> Alexei Zakharov wrote: >> > Hi, >> > >> > I have created this new thread as a single place for discussions >> > started in "Re: [testing] Peace" and "[classlib] Testing conventions = =96 >> > a proposal" threads. >> > >> > What did we have in the previous threads? >> > * Test classification proposed by Vladimir >> > * Test classification and group names proposed by George >> > * Solution for Ant and TestNG scripting issues (still being discussed) >> > >> > Since a lot of people are asking about TestNG and wanting TestNG I >> > decide to put some effort and take a closer look at this piece of >> > software. Thus during the last few days I was playing with TestNG - I >> > tried to run different kind of tests with it, to perform various >> > workloads, generate reports in different ways and etc. The purpose of >> > all this activity was to try TestNG in a real work, understand is >> > TestNG really worth our credits and how expensive can be moving to >> > TestNG from our currently implemented testing infrastructure. Now I >> > have some thoughts and facts I'd like to share with the community. >> > I've put it in the form of list for convenience. >> > >> > * TestNG works ok in normal conditions, no visible bugs >> > * It is possible to define and use various (possibly intersecting) >> > test groups with TestNG >> > * TestNG-style metadata is more convenient than JUnit test suites (now >> > I agree with this statement). IMHO this is the main TestNG benefit. >> > * It is possible to run TestNG from command line >> > * There is also the special ant task for running TestNG >> > * Not everything can be configured with the ant task or command-line >> > params, sometimes extra test suite definition file "testng.xml" is >> > needed >> > * It is possible to run jUnit tests with TestNG ("testng.xml" is >> > needed in this case) >> > * It is possible to run junit tests we currently have in Harmony with >> > TestNG without any problems and modifications of the source code. >> > However, we probably should write some number of TestNG test suite >> > definition files "testng.xml" to be able to run all our junit tests (I >> > have tried tests for bean module and some tests for luni) >> > * We can mix jUnit tests and TestNG tests in the single test suite >> > configuration =96 i.e. single testng.xml file. We can add TestNG >> > metadata to some test classes and leave other test classes untouched >> > * TestNG generates HTML reports in its own style, not a very >> > convenient one IMHO >> > * It is also possible to generate JUnitReports from the output >> > generated by TestNG >> > * Such reports will have a little bit different structure since TestNG >> > doesn't provider any information about enclosing type for test >> > methods. Names for tests (replacement for JUnit "test classes") and >> > test suites should be externally configured in "testng.xml" >> > * TestNG for Java 5 doesn't work on Harmony because some necessary >> > classes from java.util.concurrent package are missing and it seems >> > that jsr14 target (we are currently using) doesn't support annotations >> > * TestNG for Java 1.4 (javadoc version) currently works on Harmony >> > * I have half-way done script that converts TestNG 1.4 metadata >> > (javadoc) tests to TestNG 1.5 (5.0 annotations) tests. >> > >> >> Excellent summary! Thanks a lot >> >> > The question I'd like to raise now is =96 aren't we ready for TestNG >> > right now? >> I suppose we will use Java 5.0 annotations of TestNG, so it seems now we >> are not ready for TestNG. But we can continue our feasibility study, >> just like what you have done, to know if TestNG really meets our >> requirements or if there are any potential problems. Maybe we could list >> a prerequisite list. e.g, >> 1) Harmony can fully self-host TestNG with Java5 annotations >> 2) Test groups are well-defined and agreed in community >> 3) Guidelines to write TestNG testcases >> 4) Take one module to run a pilot case >> .... >> >> Please correct me if I'm wrong >> >> > For example, we could replace our harness from jUnit to >> > TestNG and lazily start converting of some failing and platform >> > dependent tests to javadoc version of TestNG. The rest of the tests >> > will remain jUnit in fact. And when our VM will be ready to handle >> > annotations we can convert all our TestNG 1.4 tests to TestNG 1.5. I >> > understand that this idea may seem to be too early. But anyway we will >> > need to change things some day since many people are unhappy with the >> > current testing infrastructure (me for example). >> Not sure if we really want to involve another migration: TestNG javadoc >> -> TestNG annotation. Any comments? >> >> > >> > Thought? Suggestions? Opposite opinions? >> > >> > With Best Regards, >> > >> > >> >> --=20 >> Richard Liang >> China Software Development Lab, IBM > > > --=20 Richard Liang China Software Development Lab, IBM=20 --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org For additional commands, e-mail: harmony-dev-help@incubator.apache.org