harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Harley <george.c.har...@googlemail.com>
Subject Re: [classlib] Layout of tests in crypto module
Date Thu, 18 May 2006 11:19:29 GMT
Hi Mikhail,

That is a very good point and your suggestion of supplementing the class 
or package name sounds like a very straightforward way around the 
problem. Because there will be a number of tests that must be in an 
identical package name to the type under test then it seems that the 
differentiating mark needs to be added to the test class name. Not 
really sure what this could be. Given that we are all settled on the 
convention that a test type has the "Test" suffix, how about just add 
"Impl" to that suffix for implementation tests giving us an "ImplTest" 
suffix.
e.g.

class under test : java.lang.SomeClassTest
implementation-independent test class : 
o.a.h.module.tests.java.lang.SomeClassTest
Harmony-specific test class : o.a.h.module.tests.java.lang.SomeClassImplTest


My 2 Euro cents...
George



Mikhail Loenko wrote:
> That sounds very reasonable, but I have a problem:
>
> I tried to implement it and found that as far as we put all test results
> into a single directory and generate a single report, we can't have
> different tests with the same name.
>
> For example we can't have impl and api tests of
> o.a.h.module.tests.java.lang.SomeClassTest
> Looks like we should put some differentiating mark to a class or 
> package name.
>
> Thanks,
> Mikhail
>
> 2006/5/18, George Harley <george.c.harley@googlemail.com>:
>> Mikhail Loenko wrote:
>> > Hi George,
>> >
>> > I use ant to build and run the tests, so I'm likely unaware of some
>> > Eclipse
>> > problems.
>> >
>> > If we put classpath test classes to
>> >   impl/java and api/java
>> > and bootclasspath ones to something like
>> >   impl/java.injected and api/java.injected
>> > will it solve the problem you describe?
>> >
>> > Thanks,
>> > Mikhail
>> >
>> >
>>
>> Hi Mikhail,
>>
>> Yes, I think that compiling to separate bin folders would do the trick ;
>> a bin for stuff that will go on the classpath and a bin for stuff that
>> will be loaded on the bootclasspath. In order to reach that goal the
>> test source itself will AFAIK need to be laid out in a similar manner.
>> That is, the directory tree will look something like the following (may
>> need to be viewed in wide-screen):
>>
>>
>> src/test/api/java             <--- implementation-independent tests
>> intended to be loaded by system class loader, compiled to bin
>>
>> src/test/api/java.injected    <--- implementation-independent tests
>> intended to be loaded by boot class loader, compiled to bin.injected
>>
>> src/test/impl/java             <--- Harmony-specific tests intended to
>> be loaded by system class loader, compiled to bin
>>
>> src/test/impl/java.injected    <--- Harmony-specific tests intended to
>> be loaded by boot class loader, compiled to bin.injected
>>
>>
>>
>> Does that sound reasonable ?
>>
>>
>> Best regards,
>> George
>>
>>
>> > 2006/5/17, George Harley <george.c.harley@googlemail.com>:
>> >> Mikhail Loenko wrote:
>> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
>> >> >> Mikhail Loenko wrote:
>> >> >> > Hi George, see below
>> >> >> >
>> >> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
>> >> >> >> Hi Mikhail,
>> >> >> >>
>> >> >> >> I have a couple of minor comments about your proposal
for a 
>> test
>> >> >> >> layouts. I should have responded sooner, I know, but I
have
>> >> suffered
>> >> >> >> from a number of hardware problems in the past few weeks
that
>> >> slowed
>> >> >> >> things down somewhat for me. Anyway, it's all great but
it 
>> would
>> >> >> be nice
>> >> >> >> to get answers to the following ...
>> >> >> >>
>> >> >> >> 1) The section on "Location of the tests in the directory
tree"
>> >> >> >> proposes <modulename>/src/tests/impl for Harmony
specific tests
>> >> and
>> >> >> >> <modulename>/src/tests/api for implementation-independent

>> tests.
>> >> >> Where
>> >> >> >> would tests go for Harmony API's that are not part of
the J2SE
>> >> >> spec but
>> >> >> >> are still accessible ? Strictly speaking they are API
as 
>> well as
>> >> >> being
>> >> >> >> specific to Harmony.
>> >> >> >
>> >> >> > The main idea is to separate tests that must pass on any 
>> conformant
>> >> >> > implementation from the tests passing on Harmony only.
>> >> >> >
>> >> >> > When these are separated we can e.g. easily validate
>> >> "implementation-
>> >> >> > independent" tests by running them against RI. Actually I

>> would not
>> >> >> > like to
>> >> >> > start an endless "philosophical" discussion here about what
are
>> >> >> > unit vs. api vs. whatever tests.
>> >> >>
>> >> >> Me ? Start a philosophical discussion ? :-)
>> >> >>
>> >> >>
>> >> >> >
>> >> >> > "api" is just a name (suggested by Tim) and might be changed

>> to any
>> >> >> > unconfusing one.
>> >> >> >
>> >> >> > So, back to your question, those must go to 'impl' as far
as
>> >> they fail
>> >> >> > on RI:
>> >> >> > "<modulename>/src/tests/impl - Harmony specific tests"
>> >> >> >
>> >> >> >> What about the location of tests designed to run on
>> >> >> >> the classpath and tests designed to run on the bootclasspath
?
>> >> >> That does
>> >> >> >> not seem to be addressed in this section. When the tests
are 
>> run
>> >> >> how are
>> >> >> >
>> >> >> > Directory defines the root of the test suite, then the package
>> >> goes,
>> >> >> > see "Package names for different types of the tests" section:
>> >> >> > "If the test is designed to be run from bootclasspath then
its
>> >> package
>> >> >> > is the same as the package of the class under test"
>> >> >> >
>> >> >>
>> >> >> Is it the case that all of the classes under a particular source
>> >> folder
>> >> >> (say src/tests/api/java) get compiled to one output folder ?
>> >> >
>> >> > We have not discuss it yet. So, suggestions are welcome!  :)
>> >> >
>> >> > Thanks,
>> >> > Mikhail
>> >> >
>> >> >
>> >>
>> >> Hi Mikhail,
>> >>
>> >>  OK, great. I'm wondering about how the test classes intended to be
>> >> loaded by the bootclasspath loader get distinguished from the classes
>> >> that are intended to be loaded by the system loader. For the sake of
>> >> discussion, imagine that all of the test classes under a given source
>> >> folder (e.g. src/tests/api/java) get compiled to just one output 
>> folder
>> >> (e.g. bin). At test runtime we only want the classes that are "in the
>> >> same package as the class under test" to be on the bootclasspath - 
>> but
>> >> how is this enforced ? If the tests are being run from an Ant script
>> >> then I think that it is possible to do this using Ant's path-like
>> >> structures where wildcards can be specified among the classpath and
>> >> bootclasspath elements. But how can an equivalent degree of 
>> separation
>> >> be enforced if the tests are being run from an IDE such as Eclipse ?
>> >>
>> >>  I am not exactly an Eclipse "power-user" so it is possible that I am
>> >> missing some piece of magic here but it appears to me that if I 
>> want to
>> >> configure the classpath and bootclasspath for running a particular 
>> set
>> >> of unit tests I can only work at the output folder level. That is, it
>> >> can be configured that classes under a root folder of "bin" are on 
>> the
>> >> classpath and that classes under the root folder "bin-other" are
>> >> similarly on the bootclasspath. But there is nothing available 
>> that only
>> >> puts classes in package java.util.* under "bin" on the 
>> bootclasspath and
>> >> everything else on the classpath.
>> >>
>> >>  All of which makes me wonder if we should aim at compiling classes
>> >> intended for the bootclasspath to one bin and classes intended for 
>> the
>> >> classpath to another bin. It wouldn't really make much difference 
>> to Ant
>> >> users but as far as I can tell it would offer more flexibility to IDE
>> >> users who would then have the potential to run the tests in a way 
>> that
>> >> was more familiar to them (runtime configuration, green bar etc).
>> >>
>> >>
>> >> Best regards,
>> >> George
>> >>
>> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> >> the "bootclasspath" and "classpath" tests distinguished
?
>> >> Purely on
>> >> >> >> package name ? Did you ever see the append I wrote to
the 
>> list a
>> >> >> couple
>> >> >> >> of weeks ago on this topic ? [1]
>> >> >> >
>> >> >> > Yes, I've seen it. As you could notice we had more test 
>> categories.
>> >> >> > They are described in the same thread. We might have
>> >> >> > "independent" tests running from bootclasspath and "specific"

>> ones
>> >> >> > running from classpath.
>> >> >> >
>> >> >> >>
>> >> >> >> 2) Still in the "Location of the tests in the directory
tree"
>> >> >> section,
>> >> >> >> shouldn't the suggested source folder names include "java"
in
>> >> there ?
>> >> >> >> e.g. <modulename>/src/tests/java/api ? What is wrong
with the
>> >> >> >> src/test/java (below here is Java code), src/test/resources

>> (below
>> >> >> here
>> >> >> >> is non-code test artefacts) convention ? I notice that
at
>> >> present the
>> >> >> >> page does not include any mention of where test resources
would
>> >> go.
>> >> >> >
>> >> >> > Yes, you are right. It's missing.
>> >> >> >
>> >> >> > It was supposed to be under test "type", like:
>> >> >> > module/src/test/api/java
>> >> >> > module/src/test/api/resources
>> >> >> >
>> >> >> >
>> >> >> >>
>> >> >> >> 3) What does the sentence "Tests are not separated by
>> >> functionality
>> >> >> >> under test, e.g. tests against clone() methods are not
>> >> separated from
>> >> >> >> tests against equals() methods" actually mean ?
>> >> >> >
>> >> >> > That was to address Tim's concern:
>> >> >> > "  > 1. Tests are separated on directory level by 'intention'
>> >> >> >
>> >> >> > I agree where the intention is broad (i.e. functional tests
vs.
>> >> >> > performance tests vs. stress tests) but I'm not convinced
that
>> >> we need
>> >> >> > to separate to a precise 'intent', at least I've never been
in a
>> >> >> > position where I've wished that my my serialization tests
are
>> >> separate
>> >> >> > from my cloning tests or whatever."
>> >> >> >
>> >> >> > Feel free to reword if something is not clear in the proposal
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Mikhail
>> >> >> >
>> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >> Best regards,
>> >> >> >> George
>> >> >> >>
>> >> >> >> [1]
>> >> >> >>
>> >> >>
>> >> 
>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200604.mbox/%3c4450A1CC.1060803@googlemail.com%3e

>>
>> >>
>> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> Mikhail Loenko wrote:
>> >> >> >> > Hello
>> >> >> >> >
>> >> >> >> > I would like to make some changes in the crypto module:
>> >> >> >> >
>> >> >> >> > - Separate implemetation-independent tests from 
>> implementation
>> >> >> >> specific
>> >> >> >> > ones.
>> >> >> >> >
>> >> >> >> > - Fix layout according to the proposed scheme [1]
>> >> >> >> >
>> >> >> >> > Please let me know if you do any changes in that
module then
>> >> >> >> > I'll delay restruct.
>> >> >> >> >
>> >> >> >> > Thanks,
>> >> >> >> > Mikhail
>> >> >> >> >
>> >> >> >> > [1]
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >> 
>> http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html 
>>
>> >>
>> >> >>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message