harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Liang <richard.lian...@gmail.com>
Subject Re: [classlib] Layout of tests in crypto module
Date Wed, 24 May 2006 08:48:27 GMT
Mikhail Loenko wrote:
> 2006/5/24, Geir Magnusson Jr <geir@pobox.com>:
>>
>>
>> George Harley wrote:
>> > Hi Mikhail,
>> >
>> > That is a very good point and your suggestion of supplementing the 
>> class
>> > or package name sounds like a very straightforward way around the
>> > problem. Because there will be a number of tests that must be in an
>> > identical package name to the type under test then it seems that the
>> > differentiating mark needs to be added to the test class name. Not
>> > really sure what this could be. Given that we are all settled on the
>> > convention that a test type has the "Test" suffix, how about just add
>> > "Impl" to that suffix for implementation tests giving us an "ImplTest"
>> > suffix.
>> > e.g.
>> >
>> > class under test : java.lang.SomeClassTest
>> > implementation-independent test class :
>> > o.a.h.module.tests.java.lang.SomeClassTest
>> > Harmony-specific test class :
>> > o.a.h.module.tests.java.lang.SomeClassImplTest
>>
>> Or
>>
>> o.a.h.<module>.apitest.java.lang.FooTest
>> o.a.h.<module>.impltest.java.lang.FooTest
>>
>>
>> Of course, the implementation test didn't need to be on the
>> bootclasspath, it would simply be
>>
>> java.lang.FooTest
>>
>> with no other garbage in the package name
>
> Hi Geir
>
> how to distinguish between bootclasspath implementation test
>   java.lang.FooTest
> and bootclasspath api test
>   java.lang.FooTest
> ?
Hello Mikhail,

Could you please explain why we need bootclasspath api test? Thanks a lot.

Best regards,
Richard
>
> Thanks,
> Mikhail
>
>
>>
>> geir
>>
>>
>> >
>> > My 2 Euro cents...
>> > George
>> >
>> >
>> >
>> > Mikhail Loenko wrote:
>> >> That sounds very reasonable, but I have a problem:
>> >>
>> >> I tried to implement it and found that as far as we put all test 
>> results
>> >> into a single directory and generate a single report, we can't have
>> >> different tests with the same name.
>> >>
>> >> For example we can't have impl and api tests of
>> >> o.a.h.module.tests.java.lang.SomeClassTest
>> >> Looks like we should put some differentiating mark to a class or
>> >> package name.
>> >>
>> >> Thanks,
>> >> Mikhail
>> >>
>> >> 2006/5/18, George Harley <george.c.harley@googlemail.com>:
>> >>> Mikhail Loenko wrote:
>> >>> > Hi George,
>> >>> >
>> >>> > I use ant to build and run the tests, so I'm likely unaware of

>> some
>> >>> > Eclipse
>> >>> > problems.
>> >>> >
>> >>> > If we put classpath test classes to
>> >>> >   impl/java and api/java
>> >>> > and bootclasspath ones to something like
>> >>> >   impl/java.injected and api/java.injected
>> >>> > will it solve the problem you describe?
>> >>> >
>> >>> > Thanks,
>> >>> > Mikhail
>> >>> >
>> >>> >
>> >>>
>> >>> Hi Mikhail,
>> >>>
>> >>> Yes, I think that compiling to separate bin folders would do the 
>> trick ;
>> >>> a bin for stuff that will go on the classpath and a bin for stuff 
>> that
>> >>> will be loaded on the bootclasspath. In order to reach that goal the
>> >>> test source itself will AFAIK need to be laid out in a similar 
>> manner.
>> >>> That is, the directory tree will look something like the 
>> following (may
>> >>> need to be viewed in wide-screen):
>> >>>
>> >>>
>> >>> src/test/api/java             <--- implementation-independent tests
>> >>> intended to be loaded by system class loader, compiled to bin
>> >>>
>> >>> src/test/api/java.injected    <--- implementation-independent tests
>> >>> intended to be loaded by boot class loader, compiled to bin.injected
>> >>>
>> >>> src/test/impl/java             <--- Harmony-specific tests 
>> intended to
>> >>> be loaded by system class loader, compiled to bin
>> >>>
>> >>> src/test/impl/java.injected    <--- Harmony-specific tests 
>> intended to
>> >>> be loaded by boot class loader, compiled to bin.injected
>> >>>
>> >>>
>> >>>
>> >>> Does that sound reasonable ?
>> >>>
>> >>>
>> >>> Best regards,
>> >>> George
>> >>>
>> >>>
>> >>> > 2006/5/17, George Harley <george.c.harley@googlemail.com>:
>> >>> >> Mikhail Loenko wrote:
>> >>> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
>> >>> >> >> Mikhail Loenko wrote:
>> >>> >> >> > Hi George, see below
>> >>> >> >> >
>> >>> >> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
>> >>> >> >> >> Hi Mikhail,
>> >>> >> >> >>
>> >>> >> >> >> I have a couple of minor comments about your
proposal for a
>> >>> test
>> >>> >> >> >> layouts. I should have responded sooner,
I know, but I have
>> >>> >> suffered
>> >>> >> >> >> from a number of hardware problems in the
past few weeks 
>> that
>> >>> >> slowed
>> >>> >> >> >> things down somewhat for me. Anyway, it's
all great but it
>> >>> would
>> >>> >> >> be nice
>> >>> >> >> >> to get answers to the following ...
>> >>> >> >> >>
>> >>> >> >> >> 1) The section on "Location of the tests
in the 
>> directory tree"
>> >>> >> >> >> proposes <modulename>/src/tests/impl
for Harmony 
>> specific tests
>> >>> >> and
>> >>> >> >> >> <modulename>/src/tests/api for implementation-independent
>> >>> tests.
>> >>> >> >> Where
>> >>> >> >> >> would tests go for Harmony API's that are
not part of 
>> the J2SE
>> >>> >> >> spec but
>> >>> >> >> >> are still accessible ? Strictly speaking
they are API as
>> >>> well as
>> >>> >> >> being
>> >>> >> >> >> specific to Harmony.
>> >>> >> >> >
>> >>> >> >> > The main idea is to separate tests that must
pass on any
>> >>> conformant
>> >>> >> >> > implementation from the tests passing on Harmony
only.
>> >>> >> >> >
>> >>> >> >> > When these are separated we can e.g. easily validate
>> >>> >> "implementation-
>> >>> >> >> > independent" tests by running them against RI.
Actually I
>> >>> would not
>> >>> >> >> > like to
>> >>> >> >> > start an endless "philosophical" discussion here
about 
>> what are
>> >>> >> >> > unit vs. api vs. whatever tests.
>> >>> >> >>
>> >>> >> >> Me ? Start a philosophical discussion ? :-)
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> >
>> >>> >> >> > "api" is just a name (suggested by Tim) and might
be changed
>> >>> to any
>> >>> >> >> > unconfusing one.
>> >>> >> >> >
>> >>> >> >> > So, back to your question, those must go to 'impl'
as far as
>> >>> >> they fail
>> >>> >> >> > on RI:
>> >>> >> >> > "<modulename>/src/tests/impl - Harmony
specific tests"
>> >>> >> >> >
>> >>> >> >> >> What about the location of tests designed
to run on
>> >>> >> >> >> the classpath and tests designed to run on
the 
>> bootclasspath ?
>> >>> >> >> That does
>> >>> >> >> >> not seem to be addressed in this section.
When the tests 
>> are
>> >>> run
>> >>> >> >> how are
>> >>> >> >> >
>> >>> >> >> > Directory defines the root of the test suite,
then the 
>> package
>> >>> >> goes,
>> >>> >> >> > see "Package names for different types of the
tests" 
>> section:
>> >>> >> >> > "If the test is designed to be run from bootclasspath

>> then its
>> >>> >> package
>> >>> >> >> > is the same as the package of the class under
test"
>> >>> >> >> >
>> >>> >> >>
>> >>> >> >> Is it the case that all of the classes under a particular

>> source
>> >>> >> folder
>> >>> >> >> (say src/tests/api/java) get compiled to one output
folder ?
>> >>> >> >
>> >>> >> > We have not discuss it yet. So, suggestions are welcome!
 :)
>> >>> >> >
>> >>> >> > Thanks,
>> >>> >> > Mikhail
>> >>> >> >
>> >>> >> >
>> >>> >>
>> >>> >> Hi Mikhail,
>> >>> >>
>> >>> >>  OK, great. I'm wondering about how the test classes intended

>> to be
>> >>> >> loaded by the bootclasspath loader get distinguished from the

>> classes
>> >>> >> that are intended to be loaded by the system loader. For the

>> sake of
>> >>> >> discussion, imagine that all of the test classes under a given

>> source
>> >>> >> folder (e.g. src/tests/api/java) get compiled to just one output
>> >>> folder
>> >>> >> (e.g. bin). At test runtime we only want the classes that are

>> "in the
>> >>> >> same package as the class under test" to be on the 
>> bootclasspath -
>> >>> but
>> >>> >> how is this enforced ? If the tests are being run from an Ant

>> script
>> >>> >> then I think that it is possible to do this using Ant's path-like
>> >>> >> structures where wildcards can be specified among the 
>> classpath and
>> >>> >> bootclasspath elements. But how can an equivalent degree of
>> >>> separation
>> >>> >> be enforced if the tests are being run from an IDE such as

>> Eclipse ?
>> >>> >>
>> >>> >>  I am not exactly an Eclipse "power-user" so it is possible

>> that I am
>> >>> >> missing some piece of magic here but it appears to me that
if I
>> >>> want to
>> >>> >> configure the classpath and bootclasspath for running a 
>> particular
>> >>> set
>> >>> >> of unit tests I can only work at the output folder level. That

>> is, it
>> >>> >> can be configured that classes under a root folder of "bin"

>> are on
>> >>> the
>> >>> >> classpath and that classes under the root folder "bin-other"
are
>> >>> >> similarly on the bootclasspath. But there is nothing available
>> >>> that only
>> >>> >> puts classes in package java.util.* under "bin" on the
>> >>> bootclasspath and
>> >>> >> everything else on the classpath.
>> >>> >>
>> >>> >>  All of which makes me wonder if we should aim at compiling

>> classes
>> >>> >> intended for the bootclasspath to one bin and classes intended

>> for
>> >>> the
>> >>> >> classpath to another bin. It wouldn't really make much difference
>> >>> to Ant
>> >>> >> users but as far as I can tell it would offer more flexibility

>> to IDE
>> >>> >> users who would then have the potential to run the tests in
a way
>> >>> that
>> >>> >> was more familiar to them (runtime configuration, green bar
etc).
>> >>> >>
>> >>> >>
>> >>> >> Best regards,
>> >>> >> George
>> >>> >>
>> >>> >> >
>> >>> >> >>
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> >> the "bootclasspath" and "classpath" tests
distinguished ?
>> >>> >> Purely on
>> >>> >> >> >> package name ? Did you ever see the append
I wrote to the
>> >>> list a
>> >>> >> >> couple
>> >>> >> >> >> of weeks ago on this topic ? [1]
>> >>> >> >> >
>> >>> >> >> > Yes, I've seen it. As you could notice we had
more test
>> >>> categories.
>> >>> >> >> > They are described in the same thread. We might
have
>> >>> >> >> > "independent" tests running from bootclasspath
and 
>> "specific"
>> >>> ones
>> >>> >> >> > running from classpath.
>> >>> >> >> >
>> >>> >> >> >>
>> >>> >> >> >> 2) Still in the "Location of the tests in
the directory 
>> tree"
>> >>> >> >> section,
>> >>> >> >> >> shouldn't the suggested source folder names
include 
>> "java" in
>> >>> >> there ?
>> >>> >> >> >> e.g. <modulename>/src/tests/java/api
? What is wrong 
>> with the
>> >>> >> >> >> src/test/java (below here is Java code),
src/test/resources
>> >>> (below
>> >>> >> >> here
>> >>> >> >> >> is non-code test artefacts) convention ?
I notice that at
>> >>> >> present the
>> >>> >> >> >> page does not include any mention of where
test 
>> resources would
>> >>> >> go.
>> >>> >> >> >
>> >>> >> >> > Yes, you are right. It's missing.
>> >>> >> >> >
>> >>> >> >> > It was supposed to be under test "type", like:
>> >>> >> >> > module/src/test/api/java
>> >>> >> >> > module/src/test/api/resources
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >>
>> >>> >> >> >> 3) What does the sentence "Tests are not
separated by
>> >>> >> functionality
>> >>> >> >> >> under test, e.g. tests against clone() methods
are not
>> >>> >> separated from
>> >>> >> >> >> tests against equals() methods" actually
mean ?
>> >>> >> >> >
>> >>> >> >> > That was to address Tim's concern:
>> >>> >> >> > "  > 1. Tests are separated on directory level
by 
>> 'intention'
>> >>> >> >> >
>> >>> >> >> > I agree where the intention is broad (i.e. functional

>> tests vs.
>> >>> >> >> > performance tests vs. stress tests) but I'm not
convinced 
>> that
>> >>> >> we need
>> >>> >> >> > to separate to a precise 'intent', at least I've
never 
>> been in a
>> >>> >> >> > position where I've wished that my my serialization
tests 
>> are
>> >>> >> separate
>> >>> >> >> > from my cloning tests or whatever."
>> >>> >> >> >
>> >>> >> >> > Feel free to reword if something is not clear
in the 
>> proposal
>> >>> >> >> >
>> >>> >> >> > Thanks,
>> >>> >> >> > Mikhail
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> Best regards,
>> >>> >> >> >> George
>> >>> >> >> >>
>> >>> >> >> >> [1]
>> >>> >> >> >>
>> >>> >> >>
>> >>> >>
>> >>> 
>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200604.mbox/%3c4450A1CC.1060803@googlemail.com%3e

>>
>> >>>
>> >>> >>
>> >>> >> >>
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> Mikhail Loenko wrote:
>> >>> >> >> >> > Hello
>> >>> >> >> >> >
>> >>> >> >> >> > I would like to make some changes in
the crypto module:
>> >>> >> >> >> >
>> >>> >> >> >> > - Separate implemetation-independent
tests from
>> >>> implementation
>> >>> >> >> >> specific
>> >>> >> >> >> > ones.
>> >>> >> >> >> >
>> >>> >> >> >> > - Fix layout according to the proposed
scheme [1]
>> >>> >> >> >> >
>> >>> >> >> >> > Please let me know if you do any changes
in that 
>> module then
>> >>> >> >> >> > I'll delay restruct.
>> >>> >> >> >> >
>> >>> >> >> >> > Thanks,
>> >>> >> >> >> > Mikhail
>> >>> >> >> >> >
>> >>> >> >> >> > [1]
>> >>> >> >> >> >
>> >>> >> >> >>
>> >>> >> >>
>> >>> >>
>> >>> 
>> http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html 
>>
>> >>>
>> >>> >>
>> >>> >> >>
>> >>>
>> >>>
>> >>> 
>> ---------------------------------------------------------------------
>> >>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >>> For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >>>
>> >>>
>> >>
>> >> ---------------------------------------------------------------------
>> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >> For additional commands, e-mail: 
>> harmony-dev-help@incubator.apache.org
>> >>
>> >>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>> >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

-- 
Richard Liang
China Software Development Lab, IBM 



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message