harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [classlib] Testing
Date Fri, 28 Apr 2006 03:39:56 GMT
Hi Paulex

2006/4/28, Paulex Yang <paulex.yang@gmail.com>:
> Mikhail Loenko wrote:
> > Then how to run only independent tests?
> >
> > Such a run would make sense to validate the tests or
> > if we switch to different RI version
> >
> It should be easy for ant script to identify independent tests by
> package name convention, e.g, we have a keyword "API" for the
> independent tests (1), and  2 can  be easily excluded.
>
> For the intended different behavior (4), test can be added to ant's
> exclude list (I don't expect there are too many such cases)

Did you count such tests? As I remember we have a lof of them in security.

If these tests are in the same package and in the same directory
then we will have something like

org.apache.harmony.security.tests.api.java.security.CodeSignerTest1 - impl
specific test methods

org.apache.harmony.security.tests.api.java.security.CodeSignerTest2 - impl
independent test methods

How do you like it?

Thanks,
Mikhail

>
> > Thanks,
> > Mikhail
> >
> > 2006/4/28, Paulex Yang <paulex.yang@gmail.com>:
> >
> >> Mikhail Loenko wrote:
> >>
> >>> And how would you split 1,2,4,5  between these two folders?
> >>>
> >>>
> >> I thought it is obvious.
> >> test 1,2,4 goes to src/test
> >> test 5 goes to src/impltest (internl/injected/boot anything is fine)
> >> and all implementation goes to src/main
> >>
> >> and they can be compiled to corresponding bin dir, i.e.
> >> bin/test
> >> bin/impltest
> >> bin/main
> >>
> >> bin/main and bin/impltest goes to bootclasspath
> >> bin/test goes to classpath
> >>
> >> Pls. correct me if I made mistake or missed sth.
> >>
> >>> Thanks,
> >>> Mikhail
> >>>
> >>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
> >>>
> >>>
> >>>> Mikhail Loenko wrote:
> >>>>
> >>>>
> >>>>> If it's separated, I'm OK with it
> >>>>>
> >>>>> Can you suggest dir names?
> >>>>>
> >>>>>
> >>>>>
> >>>> Great!
> >>>>
> >>>> Either boot or injected(as Mark said) sounds good keyword to me.  so
it
> >>>> can be one of below:
> >>>> 1. src/test.injected/
> >>>> 2. src/test/injected
> >>>> 3. src/test.boot/
> >>>> 4. src/test/boot
> >>>>
> >>>> Mmm...seems injected is more self-descriptive,  I prefer option 1. So
> >>>> the layout would be:
> >>>> src/main
> >>>> src/test
> >>>> src/test.injected
> >>>>
> >>>> ideas?
> >>>>
> >>>>
> >>>>> Thanks,
> >>>>> Mikhail
> >>>>>
> >>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
> >>>>>
> >>>>>
> >>>>>
> >>>>>> Mikhail Loenko wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> The problem is the tests will be mixed like that:
> >>>>>>>
> >>>>>>> org.apache.harmony.security.asn1 -- tests in bootclasspath
> >>>>>>> org.apache.harmony.security.cert  -- tests in bootclasspath
> >>>>>>> org.apache.harmony.security.tests -- tests in classpath
> >>>>>>> org.apache.harmony.security.x509 -- tests in bootclasspath
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> Good catch! :)
> >>>>>>
> >>>>>> It's not so hard for script to separate, if the package name
doesn't
> >>>>>> contains "test", it belongs to bootclasspath
> >>>>>>
> >>>>>> And if you concern about human reading, we can separate src
directory
> >>>>>> for them, i.e.
> >>>>>> src/main/java
> >>>>>> src/test/java
> >>>>>> src/boottest/java or src/test/boot/java
> >>>>>>
> >>>>>> comments?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> Thanks,
> >>>>>>> Mikhail
> >>>>>>>
> >>>>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> Mikhail Loenko wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Paulex,
> >>>>>>>>>
> >>>>>>>>> we have at least 8 categories of tests:
> >>>>>>>>>     running from classpath or bootclasspath
> >>>>>>>>>     implementation specific or independent
> >>>>>>>>>     testing org.apache.harmony.* or java.*
> >>>>>>>>> Could you please list how all the tests will be
named
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> Mikhail,
> >>>>>>>>
> >>>>>>>> OK, let me try. I'm not sure if all 8 categories are
necessary,
> >>>>>>>>
> >>>>>>>> 1. [classpath], [independent], [java.*]
> >>>>>>>>       o.a.h.<module name>.tests.api.<package
name>, e.g.,
> >>>>>>>> o.a.h.luni.tests.api.java.util is test package for java.util
> >>>>>>>>     I consider the module name is optional and can be
removed, so it can be
> >>>>>>>>       o.a.h.tests.api.java.util for java.util
> >>>>>>>>
> >>>>>>>> 2. [classpath], [dependent], [o.a.h.*]
> >>>>>>>>      o.a.h.<module name>.tests.<package name
without o.a.h...prefix>,
> >>>>>>>> e.g., o.a.h.luni.tests.util is test package for o.a.h.luni.util
> >>>>>>>>    Again, the module name is optional, but if module
name is not
> >>>>>>>> prefix, it should be part of package name, so it is:
> >>>>>>>>       o.a.h.tests.luni.util for o.a.h.luni.util
> >>>>>>>>
> >>>>>>>> 3. [classpath], [independent], [o.a.h.*]
> >>>>>>>>    I don't expect these classes/tests exist.
> >>>>>>>>
> >>>>>>>> 4. [classpath], [dependent], [java.*]
> >>>>>>>>    if my understanding is correct, this category means
the tests for
> >>>>>>>> intended different behavior between Harmony and RI,
I consider these
> >>>>>>>> tests should be put in same package with API tests,
and separate them by
> >>>>>>>> running script instead of different package, because
RI may fix bug in
> >>>>>>>> the revision later(e.g, Sun JDK 1.5.1 maybe?), and IIRC
in another
> >>>>>>>> thread we have agreed that we always follow latest version
of RI, so it
> >>>>>>>> will be much easier to change the different behavior
tests to API tests
> >>>>>>>> if we use script to separate them, and vice versa.
> >>>>>>>>
> >>>>>>>> 5. [bootclasspath],
> >>>>>>>>    which covers 4 categories you mentioned, the test
cases are in same
> >>>>>>>> package with implementions
> >>>>>>>>       e.g.
> >>>>>>>>       java.util is test package for java.util
> >>>>>>>>          and
> >>>>>>>>       o.a.h.luni.util is test packag for o.a.h.luni.util
> >>>>>>>>
> >>>>>>>> As a summary, we'll have four kind of test packages:
> >>>>>>>>
> >>>>>>>> a. o.a.h.<module>.tests.api.java.***  for case
1,4 above
> >>>>>>>> b. o.a.h.<module>.tests.***  for case 2 above
> >>>>>>>> c. java.***, for case 5 above
> >>>>>>>> d. o.a.h.***, for case 5 above
> >>>>>>>>
> >>>>>>>> comments?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Mikhail
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Mikhail Loenko wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> Geir,
> >>>>>>>>>>>
> >>>>>>>>>>> The problem is that no one yet suggested
a consistent solution
> >>>>>>>>>>> that would fit for all the tests and would
not get into packages like
> >>>>>>>>>>>
> >>>>>>>>>>> org.apache.harmony.security.tests.org.apache.harmony.security.util
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Everybody seems to agree that SOME implementation
specific tests are
> >>>>>>>>>>> in the same package as implementation.
> >>>>>>>>>>>
> >>>>>>>>>>> But if we have ALL the tests in the same
package as implementation,
> >>>>>>>>>>> then we can't run e.g. tests against java.beans.Beans
from classpath, so
> >>>>>>>>>>> we can't run regression test for HARMONY-358
> >>>>>>>>>>>
> >>>>>>>>>>> The question is where we put 'classpath'
 test. If we put them into
> >>>>>>>>>>> o.a.h.module.tests.[package under test]
we result in that ugly package
> >>>>>>>>>>> names like
> >>>>>>>>>>> o.a.h.security.tests.o.a.h.security.util
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> Mikhail,
> >>>>>>>>>>
> >>>>>>>>>> Why my prior proposal that o.a.h.security.tests.impl.util
is unacceptable?
> >>>>>>>>>>
> >>>>>>>>>> And I have no objection if "impl" considered
redundant, too.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Mikhail
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 2006/4/27, Geir Magnusson Jr <geir@pobox.com>:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> Geir Magnusson Jr wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> George Harley wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>  >>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>> Of course, the text module has
only "implementation-independent tests
> >>>>>>>>>>>>>> that designed to be run from
classpath". For modules that have got
> >>>>>>>>>>>>>> implementation-specific tests
then I suppose we could use something
> >>>>>>>>>>>>>> like "org.apache.harmony.[module].tests.impl.[package
under test]" or
> >>>>>>>>>>>>>> "org.apache.harmony.[module].tests.internal.[package
under test]" etc.
> >>>>>>>>>>>>>> I've got no preference.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> -1
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Put them in the same package as
the class being tested, please, if there
> >>>>>>>>>>>>> is no problem being run that way.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> Sorry to be so snippy.  Been a long
day.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Why would we want this convoluted package
name?  Why wouldn't we want to
> >>>>>>>>>>>> let them be in the same package as the
class being tested since it's
> >>>>>>>>>>>> implementation testing?
> >>>>>>>>>>>>
> >>>>>>>>>>>> geir
> >>>>>>>>>>>>
> >>>>>>>>>>>> ---------------------------------------------------------------------
> >>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>> ---------------------------------------------------------------------
> >>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Paulex Yang
> >>>>>>>>>> China Software Development Lab
> >>>>>>>>>> IBM
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> ---------------------------------------------------------------------
> >>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>> ---------------------------------------------------------------------
> >>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> --
> >>>>>>>> Paulex Yang
> >>>>>>>> China Software Development Lab
> >>>>>>>> IBM
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> ---------------------------------------------------------------------
> >>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> ---------------------------------------------------------------------
> >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> --
> >>>>>> Paulex Yang
> >>>>>> China Software Development Lab
> >>>>>> IBM
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ---------------------------------------------------------------------
> >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>> --
> >>>> Paulex Yang
> >>>> China Software Development Lab
> >>>> IBM
> >>>>
> >>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>>
> >>>>
> >>>>
> >>>>
> >>> ---------------------------------------------------------------------
> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>
> >>>
> >>>
> >>>
> >> --
> >> Paulex Yang
> >> China Software Development Lab
> >> IBM
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
> >
>
>
> --
> Paulex Yang
> China Software Development Lab
> IBM
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message