harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [classlib] Layout of tests in crypto module
Date Wed, 24 May 2006 04:30:37 GMT
2006/5/24, Geir Magnusson Jr <geir@pobox.com>:
>
>
> George Harley wrote:
> > Hi Mikhail,
> >
> > That is a very good point and your suggestion of supplementing the class
> > or package name sounds like a very straightforward way around the
> > problem. Because there will be a number of tests that must be in an
> > identical package name to the type under test then it seems that the
> > differentiating mark needs to be added to the test class name. Not
> > really sure what this could be. Given that we are all settled on the
> > convention that a test type has the "Test" suffix, how about just add
> > "Impl" to that suffix for implementation tests giving us an "ImplTest"
> > suffix.
> > e.g.
> >
> > class under test : java.lang.SomeClassTest
> > implementation-independent test class :
> > o.a.h.module.tests.java.lang.SomeClassTest
> > Harmony-specific test class :
> > o.a.h.module.tests.java.lang.SomeClassImplTest
>
> Or
>
> o.a.h.<module>.apitest.java.lang.FooTest
> o.a.h.<module>.impltest.java.lang.FooTest
>
>
> Of course, the implementation test didn't need to be on the
> bootclasspath, it would simply be
>
> java.lang.FooTest
>
> with no other garbage in the package name

Hi Geir

how to distinguish between bootclasspath implementation test
   java.lang.FooTest
and bootclasspath api test
   java.lang.FooTest
?

Thanks,
Mikhail


>
> geir
>
>
> >
> > My 2 Euro cents...
> > George
> >
> >
> >
> > Mikhail Loenko wrote:
> >> That sounds very reasonable, but I have a problem:
> >>
> >> I tried to implement it and found that as far as we put all test results
> >> into a single directory and generate a single report, we can't have
> >> different tests with the same name.
> >>
> >> For example we can't have impl and api tests of
> >> o.a.h.module.tests.java.lang.SomeClassTest
> >> Looks like we should put some differentiating mark to a class or
> >> package name.
> >>
> >> Thanks,
> >> Mikhail
> >>
> >> 2006/5/18, George Harley <george.c.harley@googlemail.com>:
> >>> Mikhail Loenko wrote:
> >>> > Hi George,
> >>> >
> >>> > I use ant to build and run the tests, so I'm likely unaware of some
> >>> > Eclipse
> >>> > problems.
> >>> >
> >>> > If we put classpath test classes to
> >>> >   impl/java and api/java
> >>> > and bootclasspath ones to something like
> >>> >   impl/java.injected and api/java.injected
> >>> > will it solve the problem you describe?
> >>> >
> >>> > Thanks,
> >>> > Mikhail
> >>> >
> >>> >
> >>>
> >>> Hi Mikhail,
> >>>
> >>> Yes, I think that compiling to separate bin folders would do the trick ;
> >>> a bin for stuff that will go on the classpath and a bin for stuff that
> >>> will be loaded on the bootclasspath. In order to reach that goal the
> >>> test source itself will AFAIK need to be laid out in a similar manner.
> >>> That is, the directory tree will look something like the following (may
> >>> need to be viewed in wide-screen):
> >>>
> >>>
> >>> src/test/api/java             <--- implementation-independent tests
> >>> intended to be loaded by system class loader, compiled to bin
> >>>
> >>> src/test/api/java.injected    <--- implementation-independent tests
> >>> intended to be loaded by boot class loader, compiled to bin.injected
> >>>
> >>> src/test/impl/java             <--- Harmony-specific tests intended to
> >>> be loaded by system class loader, compiled to bin
> >>>
> >>> src/test/impl/java.injected    <--- Harmony-specific tests intended to
> >>> be loaded by boot class loader, compiled to bin.injected
> >>>
> >>>
> >>>
> >>> Does that sound reasonable ?
> >>>
> >>>
> >>> Best regards,
> >>> George
> >>>
> >>>
> >>> > 2006/5/17, George Harley <george.c.harley@googlemail.com>:
> >>> >> Mikhail Loenko wrote:
> >>> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
> >>> >> >> Mikhail Loenko wrote:
> >>> >> >> > Hi George, see below
> >>> >> >> >
> >>> >> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
> >>> >> >> >> Hi Mikhail,
> >>> >> >> >>
> >>> >> >> >> I have a couple of minor comments about your
proposal for a
> >>> test
> >>> >> >> >> layouts. I should have responded sooner, I know,
but I have
> >>> >> suffered
> >>> >> >> >> from a number of hardware problems in the past
few weeks that
> >>> >> slowed
> >>> >> >> >> things down somewhat for me. Anyway, it's all
great but it
> >>> would
> >>> >> >> be nice
> >>> >> >> >> to get answers to the following ...
> >>> >> >> >>
> >>> >> >> >> 1) The section on "Location of the tests in the
directory tree"
> >>> >> >> >> proposes <modulename>/src/tests/impl for
Harmony specific tests
> >>> >> and
> >>> >> >> >> <modulename>/src/tests/api for implementation-independent
> >>> tests.
> >>> >> >> Where
> >>> >> >> >> would tests go for Harmony API's that are not
part of the J2SE
> >>> >> >> spec but
> >>> >> >> >> are still accessible ? Strictly speaking they
are API as
> >>> well as
> >>> >> >> being
> >>> >> >> >> specific to Harmony.
> >>> >> >> >
> >>> >> >> > The main idea is to separate tests that must pass
on any
> >>> conformant
> >>> >> >> > implementation from the tests passing on Harmony
only.
> >>> >> >> >
> >>> >> >> > When these are separated we can e.g. easily validate
> >>> >> "implementation-
> >>> >> >> > independent" tests by running them against RI. Actually
I
> >>> would not
> >>> >> >> > like to
> >>> >> >> > start an endless "philosophical" discussion here
about what are
> >>> >> >> > unit vs. api vs. whatever tests.
> >>> >> >>
> >>> >> >> Me ? Start a philosophical discussion ? :-)
> >>> >> >>
> >>> >> >>
> >>> >> >> >
> >>> >> >> > "api" is just a name (suggested by Tim) and might
be changed
> >>> to any
> >>> >> >> > unconfusing one.
> >>> >> >> >
> >>> >> >> > So, back to your question, those must go to 'impl'
as far as
> >>> >> they fail
> >>> >> >> > on RI:
> >>> >> >> > "<modulename>/src/tests/impl - Harmony specific
tests"
> >>> >> >> >
> >>> >> >> >> What about the location of tests designed to
run on
> >>> >> >> >> the classpath and tests designed to run on the
bootclasspath ?
> >>> >> >> That does
> >>> >> >> >> not seem to be addressed in this section. When
the tests are
> >>> run
> >>> >> >> how are
> >>> >> >> >
> >>> >> >> > Directory defines the root of the test suite, then
the package
> >>> >> goes,
> >>> >> >> > see "Package names for different types of the tests"
section:
> >>> >> >> > "If the test is designed to be run from bootclasspath
then its
> >>> >> package
> >>> >> >> > is the same as the package of the class under test"
> >>> >> >> >
> >>> >> >>
> >>> >> >> Is it the case that all of the classes under a particular
source
> >>> >> folder
> >>> >> >> (say src/tests/api/java) get compiled to one output folder
?
> >>> >> >
> >>> >> > We have not discuss it yet. So, suggestions are welcome! 
:)
> >>> >> >
> >>> >> > Thanks,
> >>> >> > Mikhail
> >>> >> >
> >>> >> >
> >>> >>
> >>> >> Hi Mikhail,
> >>> >>
> >>> >>  OK, great. I'm wondering about how the test classes intended to
be
> >>> >> loaded by the bootclasspath loader get distinguished from the classes
> >>> >> that are intended to be loaded by the system loader. For the sake
of
> >>> >> discussion, imagine that all of the test classes under a given
source
> >>> >> folder (e.g. src/tests/api/java) get compiled to just one output
> >>> folder
> >>> >> (e.g. bin). At test runtime we only want the classes that are "in
the
> >>> >> same package as the class under test" to be on the bootclasspath
-
> >>> but
> >>> >> how is this enforced ? If the tests are being run from an Ant script
> >>> >> then I think that it is possible to do this using Ant's path-like
> >>> >> structures where wildcards can be specified among the classpath
and
> >>> >> bootclasspath elements. But how can an equivalent degree of
> >>> separation
> >>> >> be enforced if the tests are being run from an IDE such as Eclipse
?
> >>> >>
> >>> >>  I am not exactly an Eclipse "power-user" so it is possible that
I am
> >>> >> missing some piece of magic here but it appears to me that if I
> >>> want to
> >>> >> configure the classpath and bootclasspath for running a particular
> >>> set
> >>> >> of unit tests I can only work at the output folder level. That
is, it
> >>> >> can be configured that classes under a root folder of "bin" are
on
> >>> the
> >>> >> classpath and that classes under the root folder "bin-other" are
> >>> >> similarly on the bootclasspath. But there is nothing available
> >>> that only
> >>> >> puts classes in package java.util.* under "bin" on the
> >>> bootclasspath and
> >>> >> everything else on the classpath.
> >>> >>
> >>> >>  All of which makes me wonder if we should aim at compiling classes
> >>> >> intended for the bootclasspath to one bin and classes intended
for
> >>> the
> >>> >> classpath to another bin. It wouldn't really make much difference
> >>> to Ant
> >>> >> users but as far as I can tell it would offer more flexibility
to IDE
> >>> >> users who would then have the potential to run the tests in a way
> >>> that
> >>> >> was more familiar to them (runtime configuration, green bar etc).
> >>> >>
> >>> >>
> >>> >> Best regards,
> >>> >> George
> >>> >>
> >>> >> >
> >>> >> >>
> >>> >> >>
> >>> >> >>
> >>> >> >> >> the "bootclasspath" and "classpath" tests distinguished
?
> >>> >> Purely on
> >>> >> >> >> package name ? Did you ever see the append I
wrote to the
> >>> list a
> >>> >> >> couple
> >>> >> >> >> of weeks ago on this topic ? [1]
> >>> >> >> >
> >>> >> >> > Yes, I've seen it. As you could notice we had more
test
> >>> categories.
> >>> >> >> > They are described in the same thread. We might have
> >>> >> >> > "independent" tests running from bootclasspath and
"specific"
> >>> ones
> >>> >> >> > running from classpath.
> >>> >> >> >
> >>> >> >> >>
> >>> >> >> >> 2) Still in the "Location of the tests in the
directory tree"
> >>> >> >> section,
> >>> >> >> >> shouldn't the suggested source folder names include
"java" in
> >>> >> there ?
> >>> >> >> >> e.g. <modulename>/src/tests/java/api ?
What is wrong with the
> >>> >> >> >> src/test/java (below here is Java code), src/test/resources
> >>> (below
> >>> >> >> here
> >>> >> >> >> is non-code test artefacts) convention ? I notice
that at
> >>> >> present the
> >>> >> >> >> page does not include any mention of where test
resources would
> >>> >> go.
> >>> >> >> >
> >>> >> >> > Yes, you are right. It's missing.
> >>> >> >> >
> >>> >> >> > It was supposed to be under test "type", like:
> >>> >> >> > module/src/test/api/java
> >>> >> >> > module/src/test/api/resources
> >>> >> >> >
> >>> >> >> >
> >>> >> >> >>
> >>> >> >> >> 3) What does the sentence "Tests are not separated
by
> >>> >> functionality
> >>> >> >> >> under test, e.g. tests against clone() methods
are not
> >>> >> separated from
> >>> >> >> >> tests against equals() methods" actually mean
?
> >>> >> >> >
> >>> >> >> > That was to address Tim's concern:
> >>> >> >> > "  > 1. Tests are separated on directory level
by 'intention'
> >>> >> >> >
> >>> >> >> > I agree where the intention is broad (i.e. functional
tests vs.
> >>> >> >> > performance tests vs. stress tests) but I'm not convinced
that
> >>> >> we need
> >>> >> >> > to separate to a precise 'intent', at least I've
never been in a
> >>> >> >> > position where I've wished that my my serialization
tests are
> >>> >> separate
> >>> >> >> > from my cloning tests or whatever."
> >>> >> >> >
> >>> >> >> > Feel free to reword if something is not clear in
the proposal
> >>> >> >> >
> >>> >> >> > Thanks,
> >>> >> >> > Mikhail
> >>> >> >> >
> >>> >> >> >
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >> Best regards,
> >>> >> >> >> George
> >>> >> >> >>
> >>> >> >> >> [1]
> >>> >> >> >>
> >>> >> >>
> >>> >>
> >>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200604.mbox/%3c4450A1CC.1060803@googlemail.com%3e
> >>>
> >>> >>
> >>> >> >>
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >> Mikhail Loenko wrote:
> >>> >> >> >> > Hello
> >>> >> >> >> >
> >>> >> >> >> > I would like to make some changes in the
crypto module:
> >>> >> >> >> >
> >>> >> >> >> > - Separate implemetation-independent tests
from
> >>> implementation
> >>> >> >> >> specific
> >>> >> >> >> > ones.
> >>> >> >> >> >
> >>> >> >> >> > - Fix layout according to the proposed scheme
[1]
> >>> >> >> >> >
> >>> >> >> >> > Please let me know if you do any changes
in that module then
> >>> >> >> >> > I'll delay restruct.
> >>> >> >> >> >
> >>> >> >> >> > Thanks,
> >>> >> >> >> > Mikhail
> >>> >> >> >> >
> >>> >> >> >> > [1]
> >>> >> >> >> >
> >>> >> >> >>
> >>> >> >>
> >>> >>
> >>> http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> >>>
> >>> >>
> >>> >> >>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message