harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Harley <george.c.har...@googlemail.com>
Subject Re: [classlib] Layout of tests in crypto module
Date Thu, 18 May 2006 12:34:27 GMT
Mikhail Loenko wrote:
> There are classes like X509CertImpl.java in this case impl test for it
> would be X509CertImplImplTest which does not look very good.
>
> How about ITest ?
>
> Thanks,
> Mikhail
>

Hi Mikhail,

Sure, that works for me.

Best regards,
George


> 2006/5/18, George Harley <george.c.harley@googlemail.com>:
>> Hi Mikhail,
>>
>> That is a very good point and your suggestion of supplementing the class
>> or package name sounds like a very straightforward way around the
>> problem. Because there will be a number of tests that must be in an
>> identical package name to the type under test then it seems that the
>> differentiating mark needs to be added to the test class name. Not
>> really sure what this could be. Given that we are all settled on the
>> convention that a test type has the "Test" suffix, how about just add
>> "Impl" to that suffix for implementation tests giving us an "ImplTest"
>> suffix.
>> e.g.
>>
>> class under test : java.lang.SomeClassTest
>> implementation-independent test class :
>> o.a.h.module.tests.java.lang.SomeClassTest
>> Harmony-specific test class : 
>> o.a.h.module.tests.java.lang.SomeClassImplTest
>>
>>
>> My 2 Euro cents...
>> George
>>
>>
>>
>> Mikhail Loenko wrote:
>> > That sounds very reasonable, but I have a problem:
>> >
>> > I tried to implement it and found that as far as we put all test 
>> results
>> > into a single directory and generate a single report, we can't have
>> > different tests with the same name.
>> >
>> > For example we can't have impl and api tests of
>> > o.a.h.module.tests.java.lang.SomeClassTest
>> > Looks like we should put some differentiating mark to a class or
>> > package name.
>> >
>> > Thanks,
>> > Mikhail
>> >
>> > 2006/5/18, George Harley <george.c.harley@googlemail.com>:
>> >> Mikhail Loenko wrote:
>> >> > Hi George,
>> >> >
>> >> > I use ant to build and run the tests, so I'm likely unaware of some
>> >> > Eclipse
>> >> > problems.
>> >> >
>> >> > If we put classpath test classes to
>> >> >   impl/java and api/java
>> >> > and bootclasspath ones to something like
>> >> >   impl/java.injected and api/java.injected
>> >> > will it solve the problem you describe?
>> >> >
>> >> > Thanks,
>> >> > Mikhail
>> >> >
>> >> >
>> >>
>> >> Hi Mikhail,
>> >>
>> >> Yes, I think that compiling to separate bin folders would do the 
>> trick ;
>> >> a bin for stuff that will go on the classpath and a bin for stuff 
>> that
>> >> will be loaded on the bootclasspath. In order to reach that goal the
>> >> test source itself will AFAIK need to be laid out in a similar 
>> manner.
>> >> That is, the directory tree will look something like the following 
>> (may
>> >> need to be viewed in wide-screen):
>> >>
>> >>
>> >> src/test/api/java             <--- implementation-independent tests
>> >> intended to be loaded by system class loader, compiled to bin
>> >>
>> >> src/test/api/java.injected    <--- implementation-independent tests
>> >> intended to be loaded by boot class loader, compiled to bin.injected
>> >>
>> >> src/test/impl/java             <--- Harmony-specific tests 
>> intended to
>> >> be loaded by system class loader, compiled to bin
>> >>
>> >> src/test/impl/java.injected    <--- Harmony-specific tests 
>> intended to
>> >> be loaded by boot class loader, compiled to bin.injected
>> >>
>> >>
>> >>
>> >> Does that sound reasonable ?
>> >>
>> >>
>> >> Best regards,
>> >> George
>> >>
>> >>
>> >> > 2006/5/17, George Harley <george.c.harley@googlemail.com>:
>> >> >> Mikhail Loenko wrote:
>> >> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
>> >> >> >> Mikhail Loenko wrote:
>> >> >> >> > Hi George, see below
>> >> >> >> >
>> >> >> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
>> >> >> >> >> Hi Mikhail,
>> >> >> >> >>
>> >> >> >> >> I have a couple of minor comments about your
proposal for a
>> >> test
>> >> >> >> >> layouts. I should have responded sooner, I know,
but I have
>> >> >> suffered
>> >> >> >> >> from a number of hardware problems in the past
few weeks 
>> that
>> >> >> slowed
>> >> >> >> >> things down somewhat for me. Anyway, it's all
great but it
>> >> would
>> >> >> >> be nice
>> >> >> >> >> to get answers to the following ...
>> >> >> >> >>
>> >> >> >> >> 1) The section on "Location of the tests in the
directory 
>> tree"
>> >> >> >> >> proposes <modulename>/src/tests/impl for
Harmony specific 
>> tests
>> >> >> and
>> >> >> >> >> <modulename>/src/tests/api for implementation-independent
>> >> tests.
>> >> >> >> Where
>> >> >> >> >> would tests go for Harmony API's that are not
part of the 
>> J2SE
>> >> >> >> spec but
>> >> >> >> >> are still accessible ? Strictly speaking they
are API as
>> >> well as
>> >> >> >> being
>> >> >> >> >> specific to Harmony.
>> >> >> >> >
>> >> >> >> > The main idea is to separate tests that must pass
on any
>> >> conformant
>> >> >> >> > implementation from the tests passing on Harmony
only.
>> >> >> >> >
>> >> >> >> > When these are separated we can e.g. easily validate
>> >> >> "implementation-
>> >> >> >> > independent" tests by running them against RI. Actually
I
>> >> would not
>> >> >> >> > like to
>> >> >> >> > start an endless "philosophical" discussion here
about 
>> what are
>> >> >> >> > unit vs. api vs. whatever tests.
>> >> >> >>
>> >> >> >> Me ? Start a philosophical discussion ? :-)
>> >> >> >>
>> >> >> >>
>> >> >> >> >
>> >> >> >> > "api" is just a name (suggested by Tim) and might
be changed
>> >> to any
>> >> >> >> > unconfusing one.
>> >> >> >> >
>> >> >> >> > So, back to your question, those must go to 'impl'
as far as
>> >> >> they fail
>> >> >> >> > on RI:
>> >> >> >> > "<modulename>/src/tests/impl - Harmony specific
tests"
>> >> >> >> >
>> >> >> >> >> What about the location of tests designed to
run on
>> >> >> >> >> the classpath and tests designed to run on the

>> bootclasspath ?
>> >> >> >> That does
>> >> >> >> >> not seem to be addressed in this section. When
the tests are
>> >> run
>> >> >> >> how are
>> >> >> >> >
>> >> >> >> > Directory defines the root of the test suite, then
the 
>> package
>> >> >> goes,
>> >> >> >> > see "Package names for different types of the tests"
section:
>> >> >> >> > "If the test is designed to be run from bootclasspath
then 
>> its
>> >> >> package
>> >> >> >> > is the same as the package of the class under test"
>> >> >> >> >
>> >> >> >>
>> >> >> >> Is it the case that all of the classes under a particular

>> source
>> >> >> folder
>> >> >> >> (say src/tests/api/java) get compiled to one output folder
?
>> >> >> >
>> >> >> > We have not discuss it yet. So, suggestions are welcome! 
:)
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Mikhail
>> >> >> >
>> >> >> >
>> >> >>
>> >> >> Hi Mikhail,
>> >> >>
>> >> >>  OK, great. I'm wondering about how the test classes intended 
>> to be
>> >> >> loaded by the bootclasspath loader get distinguished from the 
>> classes
>> >> >> that are intended to be loaded by the system loader. For the 
>> sake of
>> >> >> discussion, imagine that all of the test classes under a given

>> source
>> >> >> folder (e.g. src/tests/api/java) get compiled to just one output
>> >> folder
>> >> >> (e.g. bin). At test runtime we only want the classes that are 
>> "in the
>> >> >> same package as the class under test" to be on the bootclasspath
-
>> >> but
>> >> >> how is this enforced ? If the tests are being run from an Ant 
>> script
>> >> >> then I think that it is possible to do this using Ant's path-like
>> >> >> structures where wildcards can be specified among the classpath

>> and
>> >> >> bootclasspath elements. But how can an equivalent degree of
>> >> separation
>> >> >> be enforced if the tests are being run from an IDE such as 
>> Eclipse ?
>> >> >>
>> >> >>  I am not exactly an Eclipse "power-user" so it is possible 
>> that I am
>> >> >> missing some piece of magic here but it appears to me that if I
>> >> want to
>> >> >> configure the classpath and bootclasspath for running a particular
>> >> set
>> >> >> of unit tests I can only work at the output folder level. That

>> is, it
>> >> >> can be configured that classes under a root folder of "bin" are
on
>> >> the
>> >> >> classpath and that classes under the root folder "bin-other" are
>> >> >> similarly on the bootclasspath. But there is nothing available
>> >> that only
>> >> >> puts classes in package java.util.* under "bin" on the
>> >> bootclasspath and
>> >> >> everything else on the classpath.
>> >> >>
>> >> >>  All of which makes me wonder if we should aim at compiling 
>> classes
>> >> >> intended for the bootclasspath to one bin and classes intended
for
>> >> the
>> >> >> classpath to another bin. It wouldn't really make much difference
>> >> to Ant
>> >> >> users but as far as I can tell it would offer more flexibility

>> to IDE
>> >> >> users who would then have the potential to run the tests in a way
>> >> that
>> >> >> was more familiar to them (runtime configuration, green bar etc).
>> >> >>
>> >> >>
>> >> >> Best regards,
>> >> >> George
>> >> >>
>> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> >> the "bootclasspath" and "classpath" tests distinguished
?
>> >> >> Purely on
>> >> >> >> >> package name ? Did you ever see the append I
wrote to the
>> >> list a
>> >> >> >> couple
>> >> >> >> >> of weeks ago on this topic ? [1]
>> >> >> >> >
>> >> >> >> > Yes, I've seen it. As you could notice we had more
test
>> >> categories.
>> >> >> >> > They are described in the same thread. We might have
>> >> >> >> > "independent" tests running from bootclasspath and
"specific"
>> >> ones
>> >> >> >> > running from classpath.
>> >> >> >> >
>> >> >> >> >>
>> >> >> >> >> 2) Still in the "Location of the tests in the
directory 
>> tree"
>> >> >> >> section,
>> >> >> >> >> shouldn't the suggested source folder names include

>> "java" in
>> >> >> there ?
>> >> >> >> >> e.g. <modulename>/src/tests/java/api ?
What is wrong with 
>> the
>> >> >> >> >> src/test/java (below here is Java code), src/test/resources
>> >> (below
>> >> >> >> here
>> >> >> >> >> is non-code test artefacts) convention ? I notice
that at
>> >> >> present the
>> >> >> >> >> page does not include any mention of where test
resources 
>> would
>> >> >> go.
>> >> >> >> >
>> >> >> >> > Yes, you are right. It's missing.
>> >> >> >> >
>> >> >> >> > It was supposed to be under test "type", like:
>> >> >> >> > module/src/test/api/java
>> >> >> >> > module/src/test/api/resources
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >>
>> >> >> >> >> 3) What does the sentence "Tests are not separated
by
>> >> >> functionality
>> >> >> >> >> under test, e.g. tests against clone() methods
are not
>> >> >> separated from
>> >> >> >> >> tests against equals() methods" actually mean
?
>> >> >> >> >
>> >> >> >> > That was to address Tim's concern:
>> >> >> >> > "  > 1. Tests are separated on directory level
by 'intention'
>> >> >> >> >
>> >> >> >> > I agree where the intention is broad (i.e. functional

>> tests vs.
>> >> >> >> > performance tests vs. stress tests) but I'm not convinced

>> that
>> >> >> we need
>> >> >> >> > to separate to a precise 'intent', at least I've
never 
>> been in a
>> >> >> >> > position where I've wished that my my serialization
tests are
>> >> >> separate
>> >> >> >> > from my cloning tests or whatever."
>> >> >> >> >
>> >> >> >> > Feel free to reword if something is not clear in
the proposal
>> >> >> >> >
>> >> >> >> > Thanks,
>> >> >> >> > Mikhail
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> Best regards,
>> >> >> >> >> George
>> >> >> >> >>
>> >> >> >> >> [1]
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >> 
>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200604.mbox/%3c4450A1CC.1060803@googlemail.com%3e

>>
>> >>
>> >> >>
>> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> Mikhail Loenko wrote:
>> >> >> >> >> > Hello
>> >> >> >> >> >
>> >> >> >> >> > I would like to make some changes in the
crypto module:
>> >> >> >> >> >
>> >> >> >> >> > - Separate implemetation-independent tests
from
>> >> implementation
>> >> >> >> >> specific
>> >> >> >> >> > ones.
>> >> >> >> >> >
>> >> >> >> >> > - Fix layout according to the proposed scheme
[1]
>> >> >> >> >> >
>> >> >> >> >> > Please let me know if you do any changes
in that module 
>> then
>> >> >> >> >> > I'll delay restruct.
>> >> >> >> >> >
>> >> >> >> >> > Thanks,
>> >> >> >> >> > Mikhail
>> >> >> >> >> >
>> >> >> >> >> > [1]
>> >> >> >> >> >
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >> 
>> http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html 
>>
>> >>
>> >> >>
>> >> >> >>
>> >>
>> >>


---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message