harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fedotov, Alexei A" <alexei.a.fedo...@intel.com>
Subject RE: [classlib] Layout of tests in crypto module
Date Thu, 18 May 2006 12:53:36 GMT
Mikhail,

For me is hard to differentiate between X509CertImplTest and
X509CertImplITest. So I vote for inventing something different.

Why moving implementation-independent tests into a separate package
(like org.apache.harmony.test.<tested_package>) is no longer discussed?
BTW, in this case we can remove excessive Test suffix.

With best regards,
Alexei Fedotov,
Intel Middleware Products Division


>-----Original Message-----
>From: George Harley [mailto:george.c.harley@googlemail.com]
>Sent: Thursday, May 18, 2006 4:34 PM
>To: harmony-dev@incubator.apache.org
>Subject: Re: [classlib] Layout of tests in crypto module
>
>Mikhail Loenko wrote:
>> There are classes like X509CertImpl.java in this case impl test for
it
>> would be X509CertImplImplTest which does not look very good.
>>
>> How about ITest ?
>>
>> Thanks,
>> Mikhail
>>
>
>Hi Mikhail,
>
>Sure, that works for me.
>
>Best regards,
>George
>
>
>> 2006/5/18, George Harley <george.c.harley@googlemail.com>:
>>> Hi Mikhail,
>>>
>>> That is a very good point and your suggestion of supplementing the
class
>>> or package name sounds like a very straightforward way around the
>>> problem. Because there will be a number of tests that must be in an
>>> identical package name to the type under test then it seems that the
>>> differentiating mark needs to be added to the test class name. Not
>>> really sure what this could be. Given that we are all settled on the
>>> convention that a test type has the "Test" suffix, how about just
add
>>> "Impl" to that suffix for implementation tests giving us an
"ImplTest"
>>> suffix.
>>> e.g.
>>>
>>> class under test : java.lang.SomeClassTest
>>> implementation-independent test class :
>>> o.a.h.module.tests.java.lang.SomeClassTest
>>> Harmony-specific test class :
>>> o.a.h.module.tests.java.lang.SomeClassImplTest
>>>
>>>
>>> My 2 Euro cents...
>>> George
>>>
>>>
>>>
>>> Mikhail Loenko wrote:
>>> > That sounds very reasonable, but I have a problem:
>>> >
>>> > I tried to implement it and found that as far as we put all test
>>> results
>>> > into a single directory and generate a single report, we can't
have
>>> > different tests with the same name.
>>> >
>>> > For example we can't have impl and api tests of
>>> > o.a.h.module.tests.java.lang.SomeClassTest
>>> > Looks like we should put some differentiating mark to a class or
>>> > package name.
>>> >
>>> > Thanks,
>>> > Mikhail
>>> >
>>> > 2006/5/18, George Harley <george.c.harley@googlemail.com>:
>>> >> Mikhail Loenko wrote:
>>> >> > Hi George,
>>> >> >
>>> >> > I use ant to build and run the tests, so I'm likely unaware of
some
>>> >> > Eclipse
>>> >> > problems.
>>> >> >
>>> >> > If we put classpath test classes to
>>> >> >   impl/java and api/java
>>> >> > and bootclasspath ones to something like
>>> >> >   impl/java.injected and api/java.injected
>>> >> > will it solve the problem you describe?
>>> >> >
>>> >> > Thanks,
>>> >> > Mikhail
>>> >> >
>>> >> >
>>> >>
>>> >> Hi Mikhail,
>>> >>
>>> >> Yes, I think that compiling to separate bin folders would do the
>>> trick ;
>>> >> a bin for stuff that will go on the classpath and a bin for stuff
>>> that
>>> >> will be loaded on the bootclasspath. In order to reach that goal
the
>>> >> test source itself will AFAIK need to be laid out in a similar
>>> manner.
>>> >> That is, the directory tree will look something like the
following
>>> (may
>>> >> need to be viewed in wide-screen):
>>> >>
>>> >>
>>> >> src/test/api/java             <--- implementation-independent
tests
>>> >> intended to be loaded by system class loader, compiled to bin
>>> >>
>>> >> src/test/api/java.injected    <--- implementation-independent
tests
>>> >> intended to be loaded by boot class loader, compiled to
bin.injected
>>> >>
>>> >> src/test/impl/java             <--- Harmony-specific tests
>>> intended to
>>> >> be loaded by system class loader, compiled to bin
>>> >>
>>> >> src/test/impl/java.injected    <--- Harmony-specific tests
>>> intended to
>>> >> be loaded by boot class loader, compiled to bin.injected
>>> >>
>>> >>
>>> >>
>>> >> Does that sound reasonable ?
>>> >>
>>> >>
>>> >> Best regards,
>>> >> George
>>> >>
>>> >>
>>> >> > 2006/5/17, George Harley <george.c.harley@googlemail.com>:
>>> >> >> Mikhail Loenko wrote:
>>> >> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
>>> >> >> >> Mikhail Loenko wrote:
>>> >> >> >> > Hi George, see below
>>> >> >> >> >
>>> >> >> >> > 2006/5/16, George Harley
<george.c.harley@googlemail.com>:
>>> >> >> >> >> Hi Mikhail,
>>> >> >> >> >>
>>> >> >> >> >> I have a couple of minor comments about your
proposal
for a
>>> >> test
>>> >> >> >> >> layouts. I should have responded sooner,
I know, but I
have
>>> >> >> suffered
>>> >> >> >> >> from a number of hardware problems in the
past few weeks
>>> that
>>> >> >> slowed
>>> >> >> >> >> things down somewhat for me. Anyway, it's
all great but
it
>>> >> would
>>> >> >> >> be nice
>>> >> >> >> >> to get answers to the following ...
>>> >> >> >> >>
>>> >> >> >> >> 1) The section on "Location of the tests
in the
directory
>>> tree"
>>> >> >> >> >> proposes <modulename>/src/tests/impl
for Harmony
specific
>>> tests
>>> >> >> and
>>> >> >> >> >> <modulename>/src/tests/api for
implementation-independent
>>> >> tests.
>>> >> >> >> Where
>>> >> >> >> >> would tests go for Harmony API's that are
not part of
the
>>> J2SE
>>> >> >> >> spec but
>>> >> >> >> >> are still accessible ? Strictly speaking
they are API as
>>> >> well as
>>> >> >> >> being
>>> >> >> >> >> specific to Harmony.
>>> >> >> >> >
>>> >> >> >> > The main idea is to separate tests that must
pass on any
>>> >> conformant
>>> >> >> >> > implementation from the tests passing on Harmony
only.
>>> >> >> >> >
>>> >> >> >> > When these are separated we can e.g. easily validate
>>> >> >> "implementation-
>>> >> >> >> > independent" tests by running them against RI.
Actually I
>>> >> would not
>>> >> >> >> > like to
>>> >> >> >> > start an endless "philosophical" discussion here
about
>>> what are
>>> >> >> >> > unit vs. api vs. whatever tests.
>>> >> >> >>
>>> >> >> >> Me ? Start a philosophical discussion ? :-)
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> >
>>> >> >> >> > "api" is just a name (suggested by Tim) and might
be
changed
>>> >> to any
>>> >> >> >> > unconfusing one.
>>> >> >> >> >
>>> >> >> >> > So, back to your question, those must go to 'impl'
as far
as
>>> >> >> they fail
>>> >> >> >> > on RI:
>>> >> >> >> > "<modulename>/src/tests/impl - Harmony
specific tests"
>>> >> >> >> >
>>> >> >> >> >> What about the location of tests designed
to run on
>>> >> >> >> >> the classpath and tests designed to run on
the
>>> bootclasspath ?
>>> >> >> >> That does
>>> >> >> >> >> not seem to be addressed in this section.
When the tests
are
>>> >> run
>>> >> >> >> how are
>>> >> >> >> >
>>> >> >> >> > Directory defines the root of the test suite,
then the
>>> package
>>> >> >> goes,
>>> >> >> >> > see "Package names for different types of the
tests"
section:
>>> >> >> >> > "If the test is designed to be run from bootclasspath
then
>>> its
>>> >> >> package
>>> >> >> >> > is the same as the package of the class under
test"
>>> >> >> >> >
>>> >> >> >>
>>> >> >> >> Is it the case that all of the classes under a particular
>>> source
>>> >> >> folder
>>> >> >> >> (say src/tests/api/java) get compiled to one output
folder
?
>>> >> >> >
>>> >> >> > We have not discuss it yet. So, suggestions are welcome!
 :)
>>> >> >> >
>>> >> >> > Thanks,
>>> >> >> > Mikhail
>>> >> >> >
>>> >> >> >
>>> >> >>
>>> >> >> Hi Mikhail,
>>> >> >>
>>> >> >>  OK, great. I'm wondering about how the test classes intended
>>> to be
>>> >> >> loaded by the bootclasspath loader get distinguished from the
>>> classes
>>> >> >> that are intended to be loaded by the system loader. For the
>>> sake of
>>> >> >> discussion, imagine that all of the test classes under a given
>>> source
>>> >> >> folder (e.g. src/tests/api/java) get compiled to just one
output
>>> >> folder
>>> >> >> (e.g. bin). At test runtime we only want the classes that are
>>> "in the
>>> >> >> same package as the class under test" to be on the
bootclasspath -
>>> >> but
>>> >> >> how is this enforced ? If the tests are being run from an Ant
>>> script
>>> >> >> then I think that it is possible to do this using Ant's
path-like
>>> >> >> structures where wildcards can be specified among the
classpath
>>> and
>>> >> >> bootclasspath elements. But how can an equivalent degree of
>>> >> separation
>>> >> >> be enforced if the tests are being run from an IDE such as
>>> Eclipse ?
>>> >> >>
>>> >> >>  I am not exactly an Eclipse "power-user" so it is possible
>>> that I am
>>> >> >> missing some piece of magic here but it appears to me that
if
I
>>> >> want to
>>> >> >> configure the classpath and bootclasspath for running a
particular
>>> >> set
>>> >> >> of unit tests I can only work at the output folder level. That
>>> is, it
>>> >> >> can be configured that classes under a root folder of "bin"
are on
>>> >> the
>>> >> >> classpath and that classes under the root folder "bin-other"
are
>>> >> >> similarly on the bootclasspath. But there is nothing available
>>> >> that only
>>> >> >> puts classes in package java.util.* under "bin" on the
>>> >> bootclasspath and
>>> >> >> everything else on the classpath.
>>> >> >>
>>> >> >>  All of which makes me wonder if we should aim at compiling
>>> classes
>>> >> >> intended for the bootclasspath to one bin and classes intended
for
>>> >> the
>>> >> >> classpath to another bin. It wouldn't really make much
difference
>>> >> to Ant
>>> >> >> users but as far as I can tell it would offer more flexibility
>>> to IDE
>>> >> >> users who would then have the potential to run the tests in
a
way
>>> >> that
>>> >> >> was more familiar to them (runtime configuration, green bar
etc).
>>> >> >>
>>> >> >>
>>> >> >> Best regards,
>>> >> >> George
>>> >> >>
>>> >> >> >
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> >> the "bootclasspath" and "classpath" tests
distinguished
?
>>> >> >> Purely on
>>> >> >> >> >> package name ? Did you ever see the append
I wrote to
the
>>> >> list a
>>> >> >> >> couple
>>> >> >> >> >> of weeks ago on this topic ? [1]
>>> >> >> >> >
>>> >> >> >> > Yes, I've seen it. As you could notice we had
more test
>>> >> categories.
>>> >> >> >> > They are described in the same thread. We might
have
>>> >> >> >> > "independent" tests running from bootclasspath
and
"specific"
>>> >> ones
>>> >> >> >> > running from classpath.
>>> >> >> >> >
>>> >> >> >> >>
>>> >> >> >> >> 2) Still in the "Location of the tests in
the directory
>>> tree"
>>> >> >> >> section,
>>> >> >> >> >> shouldn't the suggested source folder names
include
>>> "java" in
>>> >> >> there ?
>>> >> >> >> >> e.g. <modulename>/src/tests/java/api
? What is wrong
with
>>> the
>>> >> >> >> >> src/test/java (below here is Java code),
src/test/resources
>>> >> (below
>>> >> >> >> here
>>> >> >> >> >> is non-code test artefacts) convention ?
I notice that
at
>>> >> >> present the
>>> >> >> >> >> page does not include any mention of where
test
resources
>>> would
>>> >> >> go.
>>> >> >> >> >
>>> >> >> >> > Yes, you are right. It's missing.
>>> >> >> >> >
>>> >> >> >> > It was supposed to be under test "type", like:
>>> >> >> >> > module/src/test/api/java
>>> >> >> >> > module/src/test/api/resources
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >>
>>> >> >> >> >> 3) What does the sentence "Tests are not
separated by
>>> >> >> functionality
>>> >> >> >> >> under test, e.g. tests against clone() methods
are not
>>> >> >> separated from
>>> >> >> >> >> tests against equals() methods" actually
mean ?
>>> >> >> >> >
>>> >> >> >> > That was to address Tim's concern:
>>> >> >> >> > "  > 1. Tests are separated on directory level
by
'intention'
>>> >> >> >> >
>>> >> >> >> > I agree where the intention is broad (i.e. functional
>>> tests vs.
>>> >> >> >> > performance tests vs. stress tests) but I'm not
convinced
>>> that
>>> >> >> we need
>>> >> >> >> > to separate to a precise 'intent', at least I've
never
>>> been in a
>>> >> >> >> > position where I've wished that my my serialization
tests
are
>>> >> >> separate
>>> >> >> >> > from my cloning tests or whatever."
>>> >> >> >> >
>>> >> >> >> > Feel free to reword if something is not clear
in the
proposal
>>> >> >> >> >
>>> >> >> >> > Thanks,
>>> >> >> >> > Mikhail
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> Best regards,
>>> >> >> >> >> George
>>> >> >> >> >>
>>> >> >> >> >> [1]
>>> >> >> >> >>
>>> >> >> >>
>>> >> >>
>>> >>
>>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-
>dev/200604.mbox/%3c4450A1CC.1060803@googlemail.com%3e
>>>
>>> >>
>>> >> >>
>>> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> Mikhail Loenko wrote:
>>> >> >> >> >> > Hello
>>> >> >> >> >> >
>>> >> >> >> >> > I would like to make some changes in
the crypto
module:
>>> >> >> >> >> >
>>> >> >> >> >> > - Separate implemetation-independent
tests from
>>> >> implementation
>>> >> >> >> >> specific
>>> >> >> >> >> > ones.
>>> >> >> >> >> >
>>> >> >> >> >> > - Fix layout according to the proposed
scheme [1]
>>> >> >> >> >> >
>>> >> >> >> >> > Please let me know if you do any changes
in that
module
>>> then
>>> >> >> >> >> > I'll delay restruct.
>>> >> >> >> >> >
>>> >> >> >> >> > Thanks,
>>> >> >> >> >> > Mikhail
>>> >> >> >> >> >
>>> >> >> >> >> > [1]
>>> >> >> >> >> >
>>> >> >> >> >>
>>> >> >> >>
>>> >> >>
>>> >>
>>>
>http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.
html
>>>
>>> >>
>>> >> >>
>>> >> >> >>
>>> >>
>>> >>
>
>
>---------------------------------------------------------------------
>Terms of use : http://incubator.apache.org/harmony/mailing.html
>To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>For additional commands, e-mail: harmony-dev-help@incubator.apache.org

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message