From harmony-dev-return-9959-apmail-incubator-harmony-dev-archive=incubator.apache.org@incubator.apache.org Mon Jul 10 02:52:17 2006 Return-Path: Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org Received: (qmail 73790 invoked from network); 10 Jul 2006 02:52:16 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 10 Jul 2006 02:52:16 -0000 Received: (qmail 73710 invoked by uid 500); 10 Jul 2006 02:52:12 -0000 Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org Received: (qmail 73671 invoked by uid 500); 10 Jul 2006 02:52:12 -0000 Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: harmony-dev@incubator.apache.org Delivered-To: mailing list harmony-dev@incubator.apache.org Received: (qmail 73660 invoked by uid 99); 10 Jul 2006 02:52:11 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 09 Jul 2006 19:52:11 -0700 X-ASF-Spam-Status: No, hits=1.9 required=10.0 tests=DNS_FROM_RFC_ABUSE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (asf.osuosl.org: 202.81.18.152 is neither permitted nor denied by domain of richard.liangyx@gmail.com) Received: from [202.81.18.152] (HELO ausmtp04.au.ibm.com) (202.81.18.152) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 09 Jul 2006 19:52:10 -0700 Received: from sd0208e0.au.ibm.com (d23rh904.au.ibm.com [202.81.18.202]) by ausmtp04.au.ibm.com (8.13.6/8.13.5) with ESMTP id k6A2uZDU151260 for ; Mon, 10 Jul 2006 12:56:42 +1000 Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.250.242]) by sd0208e0.au.ibm.com (8.12.10/NCO/VER6.8) with ESMTP id k6A2t1vP161104 for ; Mon, 10 Jul 2006 12:55:01 +1000 Received: from d23av01.au.ibm.com (loopback [127.0.0.1]) by d23av01.au.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id k6A2pcFZ012554 for ; Mon, 10 Jul 2006 12:51:38 +1000 Received: from d23m0011.cn.ibm.com ([9.181.32.74]) by d23av01.au.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id k6A2pSas011306 for ; Mon, 10 Jul 2006 12:51:38 +1000 Received: from [127.0.0.1] ([9.181.106.169]) by d23m0011.cn.ibm.com (Lotus Domino Release 6.5.5HF262) with ESMTP id 2006071010505267-1262 ; Mon, 10 Jul 2006 10:50:52 +0800 Message-ID: <44B1C084.3020408@gmail.com> Date: Mon, 10 Jul 2006 10:50:44 +0800 From: Richard Liang User-Agent: Thunderbird 1.5.0.4 (Windows/20060516) MIME-Version: 1.0 To: harmony-dev@incubator.apache.org Subject: Re: [classlib] Testing conventions - a proposal References: <44ABB451.30806@googlemail.com> <44AC6DF5.30100@gmail.com> <636fd28e0607060032p68896350n73c2e5e927fb090@mail.gmail.com> <44ACBE64.1000701@gmail.com> <636fd28e0607060052ke7e9acal6e2d2ee24ded7f35@mail.gmail.com> <44ACEBC4.9040500@googlemail.com> <44ADD3D2.6080008@gmail.com> <44ADDAF4.7070508@gmail.com> <44ADFE74.6030102@gmail.com> In-Reply-To: <44ADFE74.6030102@gmail.com> X-MIMETrack: Itemize by SMTP Server on D23M0011/23/M/IBM(Release 6.5.5HF262 | April 5, 2006) at 10/07/2006 10:50:53, Serialize by Router on D23M0011/23/M/IBM(Release 6.5.5HF262 | April 5, 2006) at 10/07/2006 10:51:37, Serialize complete at 10/07/2006 10:51:37 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Richard Liang wrote: > > > Paulex Yang wrote: >> Richard Liang wrote: >>> Hello All, >>> >>> After read through the document recommended by Alex, I think TestNG >>> can really meet our requirement. It provides much flexibility for >>> test configuration. ;-) >>> >>> If we decide to transfer to TestNG, we shall: >>> >>> 1. Identify Harmony testing strategy. (It's not easy) >>> 2. Define TestNG suite/groups to reflect Harmony testing strategy >>> 3. Decide to use Java 5 Annotations or Java 1.4 JavaDoc annotations >> Any difference between using 1.4 doclet or 5.0 annotation? If we use >> Java 1.4 so far, can we migrate to annotation easily? > Both 1.4 doclet and 5.0 annotation can provide same support for > testing configuration. The retention policy of TestNG's 5.0 annotation > is RUNTIME, that's the TestNG tests should be compiled into 5.0 > classes [1]. I don't think it's easy to migrate from doclet to > annotation, at least, TestNG does not this support. Correct me if I'm > wrong. ;-) > > 1. http://testng.org/doc/documentation-main.html#jdk-14 > >>> 4. Convert all JUnit tests to TestNG tests (TestNG provides a tool >>> "org.testng.JUnitConverter" for migrating from JUnit, but it seems >>> that the tool has a bug :-P ) >> I'm sorry, but...what the bug looks like? I think it is important >> because we have so many JUnit tests already, it will be a big concern >> of the TestNG solution if we have not tool to migrate. > I can show an example :-) > > For a junit tests: > > import junit.framework.TestCase; > > public class SampleTest extends TestCase{ > public void testMethod1() { > assertTrue(true); > } > } > > We suppose the corresponding TestNG test is: > > import org.testng.annotations.Test; > import static org.testng.AssertJUnit.*; > > public class SampleTest{ > @Test > public void testMethod1() { > assertTrue(true); > } > } > > But the tool will only add TestNG annotation to junit test methods: > > import org.testng.annotations.*; > import junit.framework.TestCase; > > public class SampleTest extends TestCase{ > @Test > public void testMethod1() { > assertTrue(true); > } > } Sorry Paulex, It sounds not a bug if we decide to use TestNG while still keeping the flexibility to use JUnit. Any comments? Thanks a lot. Richard > > TestNG Eclipse plugin also provide a way to convert junit test to > TestNG test[2], unfortunately, it also has bugs. :-( It should be > static import all the assert methods of org.testng.AssertJUnit, but > the converter only uses common import. > > 2. http://testng.org/doc/eclipse.html > >>> 5. Choose a module to run a pilot >>> ... >>> >>> Please correct me if I'm wrong. Thanks a lot. >>> >>> Best regards, >>> Richard. >>> >>> George Harley wrote: >>>> Alex Blewitt wrote: >>>>> On 06/07/06, Richard Liang wrote: >>>>>> >>>>>> It seems that you're very familiar with TestNG. ;-) So would you >>>>>> please >>>>>> identify what we shall do to transfer from junit to TestNG? >>>>>> Thanks a lot. >>>>> >>>>> Me? I'm just highly opinionated :-) >>>> >>>> Hi Alex, >>>> >>>> I think we are all pretty much in the TestNG novice category :-) >>>> >>>> >>>>> >>>>> There's guidelines for migrating from JUnit to TestNG at the home >>>>> page: >>>>> http://testng.org/doc/migrating.html >>>>> >>>>> "Here is a sample use that will convert all the JUnit tests in the >>>>> src/ directory to TestNG: >>>>> >>>>> java org.testng.JUnitConverter -overwrite -annotation -srcdir src" >>>>> >>>>> :-) >>>>> >>>> >>>> I have done some private experimentation with this command line >>>> utility and it seems to work well. In the first instance it would >>>> be good to preserve the JUnit "nature" of the tests - i.e. still >>>> have the test classes extend from JUnit TestCase etc - so that >>>> there is always a backwards migration path. That's me being >>>> paranoid. Note that the equivalent migration functionality in the >>>> latest TestNG plug-in for Eclipse did not allow that but, in >>>> addition to adding in the annotations, insisted on removing the >>>> inheritance from TestCase. >>>> >>>> >>>>> There's also instructions about how to set it up with an Ant-based >>>>> build: >>>>> http://testng.org/doc/ant.html >>>>> >>>>> I'll see if I can migrate the tests I've got in the Pack200 dir to >>>>> use >>>>> TestNG, so that you can see what it looks like. Unfortunately, I >>>>> doubt >>>>> that I'm going to be able to get to that much before 2 weeks time due >>>>> to other outstanding commitments ... >>>>> >>>>> Alex. >>>> >>>> Although we haven't gotten round to discussing specifics yet, it is >>>> probably timely to mention here that using the TestNG annotations >>>> approach (as opposed to the pre-1.5 Javadoc comments approach) will >>>> not work so long as we are compiling Harmony code with the "jsr14" >>>> target. It looked like the annotation metadata did not make it into >>>> the generated class files (at least this is what I saw in my own >>>> experiments). If we want to use the annotations approach we will >>>> have to wait until we move up to compiling for a 1.5 target. >>>> Hopefully that will not be too long now...... >>>> >>>> In the meantime you could try out using the Javadoc comments >>>> approach, just to get a feel for how things run. The downside to >>>> that is that your test source needs to be available at runtime so >>>> that the comments are available for the framework to examine. >>>> >>>> Best regards, >>>> George >>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>> For additional commands, e-mail: >>>>> harmony-dev-help@incubator.apache.org >>>>> >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>> >>>> >>> >> >> > -- Richard Liang China Software Development Lab, IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org For additional commands, e-mail: harmony-dev-help@incubator.apache.org