Return-Path: Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org Received: (qmail 58717 invoked from network); 28 Apr 2006 02:49:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 28 Apr 2006 02:49:29 -0000 Received: (qmail 26670 invoked by uid 500); 28 Apr 2006 02:49:24 -0000 Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org Received: (qmail 26623 invoked by uid 500); 28 Apr 2006 02:49:24 -0000 Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: harmony-dev@incubator.apache.org Delivered-To: mailing list harmony-dev@incubator.apache.org Received: (qmail 26612 invoked by uid 99); 28 Apr 2006 02:49:24 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Apr 2006 19:49:24 -0700 X-ASF-Spam-Status: No, hits=1.4 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (asf.osuosl.org: 202.81.18.152 is neither permitted nor denied by domain of paulex.yang@gmail.com) Received: from [202.81.18.152] (HELO ausmtp04.au.ibm.com) (202.81.18.152) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Apr 2006 19:49:23 -0700 Received: from sd0208e0.au.ibm.com (d23rh904.au.ibm.com [202.81.18.202]) by ausmtp04.au.ibm.com (8.13.6/8.13.5) with ESMTP id k3S2q9X2243536 for ; Fri, 28 Apr 2006 12:52:10 +1000 Received: from d23av04.au.ibm.com (d23av04.au.ibm.com [9.190.250.237]) by sd0208e0.au.ibm.com (8.12.10/NCO/VER6.8) with ESMTP id k3S2jDTR155140 for ; Fri, 28 Apr 2006 12:45:19 +1000 Received: from d23av04.au.ibm.com (loopback [127.0.0.1]) by d23av04.au.ibm.com (8.12.11/8.13.3) with ESMTP id k3S2fqln021179 for ; Fri, 28 Apr 2006 12:41:52 +1000 Received: from d23m0011.cn.ibm.com (d23m0011.cn.ibm.com [9.181.32.74]) by d23av04.au.ibm.com (8.12.11/8.12.11) with ESMTP id k3S2faev020909 for ; Fri, 28 Apr 2006 12:41:51 +1000 Received: from [127.0.0.1] ([9.181.106.184]) by d23m0011.cn.ibm.com (Lotus Domino Release 6.53HF294) with ESMTP id 2006042810413016-7990 ; Fri, 28 Apr 2006 10:41:30 +0800 Message-ID: <445180D3.209@gmail.com> Date: Fri, 28 Apr 2006 10:41:23 +0800 From: Paulex Yang User-Agent: Thunderbird 1.5.0.2 (Windows/20060308) MIME-Version: 1.0 To: harmony-dev@incubator.apache.org Subject: Re: [classlib] Testing References: <44213FE0.4070103@pobox.com> <4450414D.6050708@gmail.com> <906dd82e0604262203y504b8a0as89a3712152f8e7f6@mail.gmail.com> <44506246.9040101@gmail.com> <906dd82e0604262335i3854bcc2v2b9305c552aede7a@mail.gmail.com> <44506924.1020505@gmail.com> <906dd82e0604270113m4ce121aamc560176341a6b76f@mail.gmail.com> <44509102.7050402@gmail.com> <906dd82e0604270351l7b200081g978b7234b10f57fb@mail.gmail.com> <44517A4D.4060809@gmail.com> <906dd82e0604271932y1a32a00br2001776263a31747@mail.gmail.com> In-Reply-To: <906dd82e0604271932y1a32a00br2001776263a31747@mail.gmail.com> X-MIMETrack: Itemize by SMTP Server on D23M0011/23/M/IBM(Release 6.53HF294 | January 28, 2005) at 28/04/2006 10:41:30, Serialize by Router on D23M0011/23/M/IBM(Release 6.53HF294 | January 28, 2005) at 28/04/2006 10:41:51, Serialize complete at 28/04/2006 10:41:51 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Mikhail Loenko wrote: > Then how to run only independent tests? > > Such a run would make sense to validate the tests or > if we switch to different RI version > It should be easy for ant script to identify independent tests by package name convention, e.g, we have a keyword "API" for the independent tests (1), and 2 can be easily excluded. For the intended different behavior (4), test can be added to ant's exclude list (I don't expect there are too many such cases) > Thanks, > Mikhail > > 2006/4/28, Paulex Yang : > >> Mikhail Loenko wrote: >> >>> And how would you split 1,2,4,5 between these two folders? >>> >>> >> I thought it is obvious. >> test 1,2,4 goes to src/test >> test 5 goes to src/impltest (internl/injected/boot anything is fine) >> and all implementation goes to src/main >> >> and they can be compiled to corresponding bin dir, i.e. >> bin/test >> bin/impltest >> bin/main >> >> bin/main and bin/impltest goes to bootclasspath >> bin/test goes to classpath >> >> Pls. correct me if I made mistake or missed sth. >> >>> Thanks, >>> Mikhail >>> >>> 2006/4/27, Paulex Yang : >>> >>> >>>> Mikhail Loenko wrote: >>>> >>>> >>>>> If it's separated, I'm OK with it >>>>> >>>>> Can you suggest dir names? >>>>> >>>>> >>>>> >>>> Great! >>>> >>>> Either boot or injected(as Mark said) sounds good keyword to me. so it >>>> can be one of below: >>>> 1. src/test.injected/ >>>> 2. src/test/injected >>>> 3. src/test.boot/ >>>> 4. src/test/boot >>>> >>>> Mmm...seems injected is more self-descriptive, I prefer option 1. So >>>> the layout would be: >>>> src/main >>>> src/test >>>> src/test.injected >>>> >>>> ideas? >>>> >>>> >>>>> Thanks, >>>>> Mikhail >>>>> >>>>> 2006/4/27, Paulex Yang : >>>>> >>>>> >>>>> >>>>>> Mikhail Loenko wrote: >>>>>> >>>>>> >>>>>> >>>>>>> The problem is the tests will be mixed like that: >>>>>>> >>>>>>> org.apache.harmony.security.asn1 -- tests in bootclasspath >>>>>>> org.apache.harmony.security.cert -- tests in bootclasspath >>>>>>> org.apache.harmony.security.tests -- tests in classpath >>>>>>> org.apache.harmony.security.x509 -- tests in bootclasspath >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> Good catch! :) >>>>>> >>>>>> It's not so hard for script to separate, if the package name doesn't >>>>>> contains "test", it belongs to bootclasspath >>>>>> >>>>>> And if you concern about human reading, we can separate src directory >>>>>> for them, i.e. >>>>>> src/main/java >>>>>> src/test/java >>>>>> src/boottest/java or src/test/boot/java >>>>>> >>>>>> comments? >>>>>> >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> Mikhail >>>>>>> >>>>>>> 2006/4/27, Paulex Yang : >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Mikhail Loenko wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Paulex, >>>>>>>>> >>>>>>>>> we have at least 8 categories of tests: >>>>>>>>> running from classpath or bootclasspath >>>>>>>>> implementation specific or independent >>>>>>>>> testing org.apache.harmony.* or java.* >>>>>>>>> Could you please list how all the tests will be named >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> Mikhail, >>>>>>>> >>>>>>>> OK, let me try. I'm not sure if all 8 categories are necessary, >>>>>>>> >>>>>>>> 1. [classpath], [independent], [java.*] >>>>>>>> o.a.h..tests.api., e.g., >>>>>>>> o.a.h.luni.tests.api.java.util is test package for java.util >>>>>>>> I consider the module name is optional and can be removed, so it can be >>>>>>>> o.a.h.tests.api.java.util for java.util >>>>>>>> >>>>>>>> 2. [classpath], [dependent], [o.a.h.*] >>>>>>>> o.a.h..tests., >>>>>>>> e.g., o.a.h.luni.tests.util is test package for o.a.h.luni.util >>>>>>>> Again, the module name is optional, but if module name is not >>>>>>>> prefix, it should be part of package name, so it is: >>>>>>>> o.a.h.tests.luni.util for o.a.h.luni.util >>>>>>>> >>>>>>>> 3. [classpath], [independent], [o.a.h.*] >>>>>>>> I don't expect these classes/tests exist. >>>>>>>> >>>>>>>> 4. [classpath], [dependent], [java.*] >>>>>>>> if my understanding is correct, this category means the tests for >>>>>>>> intended different behavior between Harmony and RI, I consider these >>>>>>>> tests should be put in same package with API tests, and separate them by >>>>>>>> running script instead of different package, because RI may fix bug in >>>>>>>> the revision later(e.g, Sun JDK 1.5.1 maybe?), and IIRC in another >>>>>>>> thread we have agreed that we always follow latest version of RI, so it >>>>>>>> will be much easier to change the different behavior tests to API tests >>>>>>>> if we use script to separate them, and vice versa. >>>>>>>> >>>>>>>> 5. [bootclasspath], >>>>>>>> which covers 4 categories you mentioned, the test cases are in same >>>>>>>> package with implementions >>>>>>>> e.g. >>>>>>>> java.util is test package for java.util >>>>>>>> and >>>>>>>> o.a.h.luni.util is test packag for o.a.h.luni.util >>>>>>>> >>>>>>>> As a summary, we'll have four kind of test packages: >>>>>>>> >>>>>>>> a. o.a.h..tests.api.java.*** for case 1,4 above >>>>>>>> b. o.a.h..tests.*** for case 2 above >>>>>>>> c. java.***, for case 5 above >>>>>>>> d. o.a.h.***, for case 5 above >>>>>>>> >>>>>>>> comments? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Mikhail >>>>>>>>> >>>>>>>>> >>>>>>>>> 2006/4/27, Paulex Yang : >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> Mikhail Loenko wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Geir, >>>>>>>>>>> >>>>>>>>>>> The problem is that no one yet suggested a consistent solution >>>>>>>>>>> that would fit for all the tests and would not get into packages like >>>>>>>>>>> >>>>>>>>>>> org.apache.harmony.security.tests.org.apache.harmony.security.util >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Everybody seems to agree that SOME implementation specific tests are >>>>>>>>>>> in the same package as implementation. >>>>>>>>>>> >>>>>>>>>>> But if we have ALL the tests in the same package as implementation, >>>>>>>>>>> then we can't run e.g. tests against java.beans.Beans from classpath, so >>>>>>>>>>> we can't run regression test for HARMONY-358 >>>>>>>>>>> >>>>>>>>>>> The question is where we put 'classpath' test. If we put them into >>>>>>>>>>> o.a.h.module.tests.[package under test] we result in that ugly package >>>>>>>>>>> names like >>>>>>>>>>> o.a.h.security.tests.o.a.h.security.util >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Mikhail, >>>>>>>>>> >>>>>>>>>> Why my prior proposal that o.a.h.security.tests.impl.util is unacceptable? >>>>>>>>>> >>>>>>>>>> And I have no objection if "impl" considered redundant, too. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Mikhail >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 2006/4/27, Geir Magnusson Jr : >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Geir Magnusson Jr wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> George Harley wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>> Of course, the text module has only "implementation-independent tests >>>>>>>>>>>>>> that designed to be run from classpath". For modules that have got >>>>>>>>>>>>>> implementation-specific tests then I suppose we could use something >>>>>>>>>>>>>> like "org.apache.harmony.[module].tests.impl.[package under test]" or >>>>>>>>>>>>>> "org.apache.harmony.[module].tests.internal.[package under test]" etc. >>>>>>>>>>>>>> I've got no preference. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> -1 >>>>>>>>>>>>> >>>>>>>>>>>>> Put them in the same package as the class being tested, please, if there >>>>>>>>>>>>> is no problem being run that way. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> Sorry to be so snippy. Been a long day. >>>>>>>>>>>> >>>>>>>>>>>> Why would we want this convoluted package name? Why wouldn't we want to >>>>>>>>>>>> let them be in the same package as the class being tested since it's >>>>>>>>>>>> implementation testing? >>>>>>>>>>>> >>>>>>>>>>>> geir >>>>>>>>>>>> >>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Paulex Yang >>>>>>>>>> China Software Development Lab >>>>>>>>>> IBM >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> -- >>>>>>>> Paulex Yang >>>>>>>> China Software Development Lab >>>>>>>> IBM >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> -- >>>>>> Paulex Yang >>>>>> China Software Development Lab >>>>>> IBM >>>>>> >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> --------------------------------------------------------------------- >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>>> >>>>> >>>>> >>>>> >>>>> >>>> -- >>>> Paulex Yang >>>> China Software Development Lab >>>> IBM >>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>>> >>>> >>>> >>>> >>> --------------------------------------------------------------------- >>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >>> >>> >>> >>> >> -- >> Paulex Yang >> China Software Development Lab >> IBM >> >> >> >> --------------------------------------------------------------------- >> Terms of use : http://incubator.apache.org/harmony/mailing.html >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org >> >> >> > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > For additional commands, e-mail: harmony-dev-help@incubator.apache.org > > > -- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org For additional commands, e-mail: harmony-dev-help@incubator.apache.org