Return-Path: Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org Received: (qmail 78360 invoked from network); 28 Apr 2006 03:40:33 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 28 Apr 2006 03:40:33 -0000 Received: (qmail 74077 invoked by uid 500); 28 Apr 2006 03:40:21 -0000 Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org Received: (qmail 74022 invoked by uid 500); 28 Apr 2006 03:40:21 -0000 Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: harmony-dev@incubator.apache.org Delivered-To: mailing list harmony-dev@incubator.apache.org Received: (qmail 74011 invoked by uid 99); 28 Apr 2006 03:40:21 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Apr 2006 20:40:21 -0700 X-ASF-Spam-Status: No, hits=1.3 required=10.0 tests=RCVD_IN_BL_SPAMCOP_NET,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of mloenko@gmail.com designates 64.233.182.185 as permitted sender) Received: from [64.233.182.185] (HELO nproxy.gmail.com) (64.233.182.185) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Apr 2006 20:40:20 -0700 Received: by nproxy.gmail.com with SMTP id p48so1510666nfa for ; Thu, 27 Apr 2006 20:39:58 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=BaoQPx8XlM/hu6cWkoW1JgUVVwMTb0Ye8RZYgwTlyIagAnTEyjNUt3MQrurxYKezCQQ34BwHHkmKawbm38gcNsL5M3+OxWxVLVS3lvgHOy2kejxxC49MzBb9dzVI5wonSRe1JD9xW19ujW+owJNFrYiJGq8mLefH0/quhf0KqP0= Received: by 10.49.18.10 with SMTP id v10mr930072nfi; Thu, 27 Apr 2006 20:39:56 -0700 (PDT) Received: by 10.49.6.5 with HTTP; Thu, 27 Apr 2006 20:39:56 -0700 (PDT) Message-ID: <906dd82e0604272039s46e4a246g5668d4280e203417@mail.gmail.com> Date: Fri, 28 Apr 2006 10:39:56 +0700 From: "Mikhail Loenko" To: harmony-dev@incubator.apache.org Subject: Re: [classlib] Testing In-Reply-To: <445180D3.209@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <44213FE0.4070103@pobox.com> <44506246.9040101@gmail.com> <906dd82e0604262335i3854bcc2v2b9305c552aede7a@mail.gmail.com> <44506924.1020505@gmail.com> <906dd82e0604270113m4ce121aamc560176341a6b76f@mail.gmail.com> <44509102.7050402@gmail.com> <906dd82e0604270351l7b200081g978b7234b10f57fb@mail.gmail.com> <44517A4D.4060809@gmail.com> <906dd82e0604271932y1a32a00br2001776263a31747@mail.gmail.com> <445180D3.209@gmail.com> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Hi Paulex 2006/4/28, Paulex Yang : > Mikhail Loenko wrote: > > Then how to run only independent tests? > > > > Such a run would make sense to validate the tests or > > if we switch to different RI version > > > It should be easy for ant script to identify independent tests by > package name convention, e.g, we have a keyword "API" for the > independent tests (1), and 2 can be easily excluded. > > For the intended different behavior (4), test can be added to ant's > exclude list (I don't expect there are too many such cases) Did you count such tests? As I remember we have a lof of them in security. If these tests are in the same package and in the same directory then we will have something like org.apache.harmony.security.tests.api.java.security.CodeSignerTest1 - impl specific test methods org.apache.harmony.security.tests.api.java.security.CodeSignerTest2 - impl independent test methods How do you like it? Thanks, Mikhail > > > Thanks, > > Mikhail > > > > 2006/4/28, Paulex Yang : > > > >> Mikhail Loenko wrote: > >> > >>> And how would you split 1,2,4,5 between these two folders? > >>> > >>> > >> I thought it is obvious. > >> test 1,2,4 goes to src/test > >> test 5 goes to src/impltest (internl/injected/boot anything is fine) > >> and all implementation goes to src/main > >> > >> and they can be compiled to corresponding bin dir, i.e. > >> bin/test > >> bin/impltest > >> bin/main > >> > >> bin/main and bin/impltest goes to bootclasspath > >> bin/test goes to classpath > >> > >> Pls. correct me if I made mistake or missed sth. > >> > >>> Thanks, > >>> Mikhail > >>> > >>> 2006/4/27, Paulex Yang : > >>> > >>> > >>>> Mikhail Loenko wrote: > >>>> > >>>> > >>>>> If it's separated, I'm OK with it > >>>>> > >>>>> Can you suggest dir names? > >>>>> > >>>>> > >>>>> > >>>> Great! > >>>> > >>>> Either boot or injected(as Mark said) sounds good keyword to me. so= it > >>>> can be one of below: > >>>> 1. src/test.injected/ > >>>> 2. src/test/injected > >>>> 3. src/test.boot/ > >>>> 4. src/test/boot > >>>> > >>>> Mmm...seems injected is more self-descriptive, I prefer option 1. S= o > >>>> the layout would be: > >>>> src/main > >>>> src/test > >>>> src/test.injected > >>>> > >>>> ideas? > >>>> > >>>> > >>>>> Thanks, > >>>>> Mikhail > >>>>> > >>>>> 2006/4/27, Paulex Yang : > >>>>> > >>>>> > >>>>> > >>>>>> Mikhail Loenko wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>>> The problem is the tests will be mixed like that: > >>>>>>> > >>>>>>> org.apache.harmony.security.asn1 -- tests in bootclasspath > >>>>>>> org.apache.harmony.security.cert -- tests in bootclasspath > >>>>>>> org.apache.harmony.security.tests -- tests in classpath > >>>>>>> org.apache.harmony.security.x509 -- tests in bootclasspath > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> Good catch! :) > >>>>>> > >>>>>> It's not so hard for script to separate, if the package name doesn= 't > >>>>>> contains "test", it belongs to bootclasspath > >>>>>> > >>>>>> And if you concern about human reading, we can separate src direct= ory > >>>>>> for them, i.e. > >>>>>> src/main/java > >>>>>> src/test/java > >>>>>> src/boottest/java or src/test/boot/java > >>>>>> > >>>>>> comments? > >>>>>> > >>>>>> > >>>>>> > >>>>>>> Thanks, > >>>>>>> Mikhail > >>>>>>> > >>>>>>> 2006/4/27, Paulex Yang : > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> Mikhail Loenko wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> Paulex, > >>>>>>>>> > >>>>>>>>> we have at least 8 categories of tests: > >>>>>>>>> running from classpath or bootclasspath > >>>>>>>>> implementation specific or independent > >>>>>>>>> testing org.apache.harmony.* or java.* > >>>>>>>>> Could you please list how all the tests will be named > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> Mikhail, > >>>>>>>> > >>>>>>>> OK, let me try. I'm not sure if all 8 categories are necessary, > >>>>>>>> > >>>>>>>> 1. [classpath], [independent], [java.*] > >>>>>>>> o.a.h..tests.api., e.g., > >>>>>>>> o.a.h.luni.tests.api.java.util is test package for java.util > >>>>>>>> I consider the module name is optional and can be removed, s= o it can be > >>>>>>>> o.a.h.tests.api.java.util for java.util > >>>>>>>> > >>>>>>>> 2. [classpath], [dependent], [o.a.h.*] > >>>>>>>> o.a.h..tests., > >>>>>>>> e.g., o.a.h.luni.tests.util is test package for o.a.h.luni.util > >>>>>>>> Again, the module name is optional, but if module name is not > >>>>>>>> prefix, it should be part of package name, so it is: > >>>>>>>> o.a.h.tests.luni.util for o.a.h.luni.util > >>>>>>>> > >>>>>>>> 3. [classpath], [independent], [o.a.h.*] > >>>>>>>> I don't expect these classes/tests exist. > >>>>>>>> > >>>>>>>> 4. [classpath], [dependent], [java.*] > >>>>>>>> if my understanding is correct, this category means the tests= for > >>>>>>>> intended different behavior between Harmony and RI, I consider t= hese > >>>>>>>> tests should be put in same package with API tests, and separate= them by > >>>>>>>> running script instead of different package, because RI may fix = bug in > >>>>>>>> the revision later(e.g, Sun JDK 1.5.1 maybe?), and IIRC in anoth= er > >>>>>>>> thread we have agreed that we always follow latest version of RI= , so it > >>>>>>>> will be much easier to change the different behavior tests to AP= I tests > >>>>>>>> if we use script to separate them, and vice versa. > >>>>>>>> > >>>>>>>> 5. [bootclasspath], > >>>>>>>> which covers 4 categories you mentioned, the test cases are i= n same > >>>>>>>> package with implementions > >>>>>>>> e.g. > >>>>>>>> java.util is test package for java.util > >>>>>>>> and > >>>>>>>> o.a.h.luni.util is test packag for o.a.h.luni.util > >>>>>>>> > >>>>>>>> As a summary, we'll have four kind of test packages: > >>>>>>>> > >>>>>>>> a. o.a.h..tests.api.java.*** for case 1,4 above > >>>>>>>> b. o.a.h..tests.*** for case 2 above > >>>>>>>> c. java.***, for case 5 above > >>>>>>>> d. o.a.h.***, for case 5 above > >>>>>>>> > >>>>>>>> comments? > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Mikhail > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> 2006/4/27, Paulex Yang : > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> Mikhail Loenko wrote: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> Geir, > >>>>>>>>>>> > >>>>>>>>>>> The problem is that no one yet suggested a consistent solutio= n > >>>>>>>>>>> that would fit for all the tests and would not get into packa= ges like > >>>>>>>>>>> > >>>>>>>>>>> org.apache.harmony.security.tests.org.apache.harmony.security= .util > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Everybody seems to agree that SOME implementation specific te= sts are > >>>>>>>>>>> in the same package as implementation. > >>>>>>>>>>> > >>>>>>>>>>> But if we have ALL the tests in the same package as implement= ation, > >>>>>>>>>>> then we can't run e.g. tests against java.beans.Beans from cl= asspath, so > >>>>>>>>>>> we can't run regression test for HARMONY-358 > >>>>>>>>>>> > >>>>>>>>>>> The question is where we put 'classpath' test. If we put the= m into > >>>>>>>>>>> o.a.h.module.tests.[package under test] we result in that ugl= y package > >>>>>>>>>>> names like > >>>>>>>>>>> o.a.h.security.tests.o.a.h.security.util > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> Mikhail, > >>>>>>>>>> > >>>>>>>>>> Why my prior proposal that o.a.h.security.tests.impl.util is u= nacceptable? > >>>>>>>>>> > >>>>>>>>>> And I have no objection if "impl" considered redundant, too. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> Mikhail > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> 2006/4/27, Geir Magnusson Jr : > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> Geir Magnusson Jr wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>> George Harley wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> >> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>>> Of course, the text module has only "implementation-indepe= ndent tests > >>>>>>>>>>>>>> that designed to be run from classpath". For modules that = have got > >>>>>>>>>>>>>> implementation-specific tests then I suppose we could use = something > >>>>>>>>>>>>>> like "org.apache.harmony.[module].tests.impl.[package unde= r test]" or > >>>>>>>>>>>>>> "org.apache.harmony.[module].tests.internal.[package under= test]" etc. > >>>>>>>>>>>>>> I've got no preference. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> -1 > >>>>>>>>>>>>> > >>>>>>>>>>>>> Put them in the same package as the class being tested, ple= ase, if there > >>>>>>>>>>>>> is no problem being run that way. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> Sorry to be so snippy. Been a long day. > >>>>>>>>>>>> > >>>>>>>>>>>> Why would we want this convoluted package name? Why wouldn'= t we want to > >>>>>>>>>>>> let them be in the same package as the class being tested si= nce it's > >>>>>>>>>>>> implementation testing? > >>>>>>>>>>>> > >>>>>>>>>>>> geir > >>>>>>>>>>>> > >>>>>>>>>>>> ------------------------------------------------------------= --------- > >>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.h= tml > >>>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.ap= ache.org > >>>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.= apache.org > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> -------------------------------------------------------------= -------- > >>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.ht= ml > >>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apa= che.org > >>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.a= pache.org > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Paulex Yang > >>>>>>>>>> China Software Development Lab > >>>>>>>>>> IBM > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> --------------------------------------------------------------= ------- > >>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.htm= l > >>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apac= he.org > >>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.ap= ache.org > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> ---------------------------------------------------------------= ------ > >>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apach= e.org > >>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apa= che.org > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> -- > >>>>>>>> Paulex Yang > >>>>>>>> China Software Development Lab > >>>>>>>> IBM > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> ----------------------------------------------------------------= ----- > >>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache= .org > >>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apac= he.org > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> -----------------------------------------------------------------= ---- > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.= org > >>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apach= e.org > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> -- > >>>>>> Paulex Yang > >>>>>> China Software Development Lab > >>>>>> IBM > >>>>>> > >>>>>> > >>>>>> > >>>>>> ------------------------------------------------------------------= --- > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.o= rg > >>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache= .org > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> -------------------------------------------------------------------= -- > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.or= g > >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.= org > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>> -- > >>>> Paulex Yang > >>>> China Software Development Lab > >>>> IBM > >>>> > >>>> > >>>> > >>>> --------------------------------------------------------------------= - > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.o= rg > >>>> > >>>> > >>>> > >>>> > >>> --------------------------------------------------------------------- > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.or= g > >>> > >>> > >>> > >>> > >> -- > >> Paulex Yang > >> China Software Development Lab > >> IBM > >> > >> > >> > >> --------------------------------------------------------------------- > >> Terms of use : http://incubator.apache.org/harmony/mailing.html > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org > >> > >> > >> > > > > --------------------------------------------------------------------- > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org > > > > > > > > > -- > Paulex Yang > China Software Development Lab > IBM > > > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org > For additional commands, e-mail: harmony-dev-help@incubator.apache.org > > --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org For additional commands, e-mail: harmony-dev-help@incubator.apache.org