harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stepan Mishura" <stepan.mish...@gmail.com>
Subject Re: [classlib] Layout of tests in crypto module
Date Thu, 18 May 2006 12:49:16 GMT
It is hard to notice 'I' letter between others :-) , for example, the next
doesn't look good for me:
SomeExceptionITest.java, SomeExceptionTest.java

What about SomeException_ImplTest.java?

Thanks,
Stepan.

On 5/18/06, Mikhail Loenko wrote:
>
> There are classes like X509CertImpl.java in this case impl test for it
> would be X509CertImplImplTest which does not look very good.
>
> How about ITest ?
>
> Thanks,
> Mikhail
>
> 2006/5/18, George Harley <george.c.harley@googlemail.com>:
> > Hi Mikhail,
> >
> > That is a very good point and your suggestion of supplementing the class
> > or package name sounds like a very straightforward way around the
> > problem. Because there will be a number of tests that must be in an
> > identical package name to the type under test then it seems that the
> > differentiating mark needs to be added to the test class name. Not
> > really sure what this could be. Given that we are all settled on the
> > convention that a test type has the "Test" suffix, how about just add
> > "Impl" to that suffix for implementation tests giving us an "ImplTest"
> > suffix.
> > e.g.
> >
> > class under test : java.lang.SomeClassTest
> > implementation-independent test class :
> > o.a.h.module.tests.java.lang.SomeClassTest
> > Harmony-specific test class :
> o.a.h.module.tests.java.lang.SomeClassImplTest
> >
> >
> > My 2 Euro cents...
> > George
> >
> >
> >
> > Mikhail Loenko wrote:
> > > That sounds very reasonable, but I have a problem:
> > >
> > > I tried to implement it and found that as far as we put all test
> results
> > > into a single directory and generate a single report, we can't have
> > > different tests with the same name.
> > >
> > > For example we can't have impl and api tests of
> > > o.a.h.module.tests.java.lang.SomeClassTest
> > > Looks like we should put some differentiating mark to a class or
> > > package name.
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > 2006/5/18, George Harley <george.c.harley@googlemail.com>:
> > >> Mikhail Loenko wrote:
> > >> > Hi George,
> > >> >
> > >> > I use ant to build and run the tests, so I'm likely unaware of some
> > >> > Eclipse
> > >> > problems.
> > >> >
> > >> > If we put classpath test classes to
> > >> >   impl/java and api/java
> > >> > and bootclasspath ones to something like
> > >> >   impl/java.injected and api/java.injected
> > >> > will it solve the problem you describe?
> > >> >
> > >> > Thanks,
> > >> > Mikhail
> > >> >
> > >> >
> > >>
> > >> Hi Mikhail,
> > >>
> > >> Yes, I think that compiling to separate bin folders would do the
> trick ;
> > >> a bin for stuff that will go on the classpath and a bin for stuff
> that
> > >> will be loaded on the bootclasspath. In order to reach that goal the
> > >> test source itself will AFAIK need to be laid out in a similar
> manner.
> > >> That is, the directory tree will look something like the following
> (may
> > >> need to be viewed in wide-screen):
> > >>
> > >>
> > >> src/test/api/java             <--- implementation-independent tests
> > >> intended to be loaded by system class loader, compiled to bin
> > >>
> > >> src/test/api/java.injected    <--- implementation-independent tests
> > >> intended to be loaded by boot class loader, compiled to bin.injected
> > >>
> > >> src/test/impl/java             <--- Harmony-specific tests intended
> to
> > >> be loaded by system class loader, compiled to bin
> > >>
> > >> src/test/impl/java.injected    <--- Harmony-specific tests intended
> to
> > >> be loaded by boot class loader, compiled to bin.injected
> > >>
> > >>
> > >>
> > >> Does that sound reasonable ?
> > >>
> > >>
> > >> Best regards,
> > >> George
> > >>
> > >>
> > >> > 2006/5/17, George Harley <george.c.harley@googlemail.com>:
> > >> >> Mikhail Loenko wrote:
> > >> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
> > >> >> >> Mikhail Loenko wrote:
> > >> >> >> > Hi George, see below
> > >> >> >> >
> > >> >> >> > 2006/5/16, George Harley <george.c.harley@googlemail.com>:
> > >> >> >> >> Hi Mikhail,
> > >> >> >> >>
> > >> >> >> >> I have a couple of minor comments about your
proposal for a
> > >> test
> > >> >> >> >> layouts. I should have responded sooner, I know,
but I have
> > >> >> suffered
> > >> >> >> >> from a number of hardware problems in the past
few weeks
> that
> > >> >> slowed
> > >> >> >> >> things down somewhat for me. Anyway, it's all
great but it
> > >> would
> > >> >> >> be nice
> > >> >> >> >> to get answers to the following ...
> > >> >> >> >>
> > >> >> >> >> 1) The section on "Location of the tests in
the directory
> tree"
> > >> >> >> >> proposes <modulename>/src/tests/impl for
Harmony specific
> tests
> > >> >> and
> > >> >> >> >> <modulename>/src/tests/api for implementation-independent
> > >> tests.
> > >> >> >> Where
> > >> >> >> >> would tests go for Harmony API's that are not
part of the
> J2SE
> > >> >> >> spec but
> > >> >> >> >> are still accessible ? Strictly speaking they
are API as
> > >> well as
> > >> >> >> being
> > >> >> >> >> specific to Harmony.
> > >> >> >> >
> > >> >> >> > The main idea is to separate tests that must pass
on any
> > >> conformant
> > >> >> >> > implementation from the tests passing on Harmony
only.
> > >> >> >> >
> > >> >> >> > When these are separated we can e.g. easily validate
> > >> >> "implementation-
> > >> >> >> > independent" tests by running them against RI. Actually
I
> > >> would not
> > >> >> >> > like to
> > >> >> >> > start an endless "philosophical" discussion here
about what
> are
> > >> >> >> > unit vs. api vs. whatever tests.
> > >> >> >>
> > >> >> >> Me ? Start a philosophical discussion ? :-)
> > >> >> >>
> > >> >> >>
> > >> >> >> >
> > >> >> >> > "api" is just a name (suggested by Tim) and might
be changed
> > >> to any
> > >> >> >> > unconfusing one.
> > >> >> >> >
> > >> >> >> > So, back to your question, those must go to 'impl'
as far as
> > >> >> they fail
> > >> >> >> > on RI:
> > >> >> >> > "<modulename>/src/tests/impl - Harmony specific
tests"
> > >> >> >> >
> > >> >> >> >> What about the location of tests designed to
run on
> > >> >> >> >> the classpath and tests designed to run on the
bootclasspath
> ?
> > >> >> >> That does
> > >> >> >> >> not seem to be addressed in this section. When
the tests are
> > >> run
> > >> >> >> how are
> > >> >> >> >
> > >> >> >> > Directory defines the root of the test suite, then
the
> package
> > >> >> goes,
> > >> >> >> > see "Package names for different types of the tests"
section:
> > >> >> >> > "If the test is designed to be run from bootclasspath
then
> its
> > >> >> package
> > >> >> >> > is the same as the package of the class under test"
> > >> >> >> >
> > >> >> >>
> > >> >> >> Is it the case that all of the classes under a particular
> source
> > >> >> folder
> > >> >> >> (say src/tests/api/java) get compiled to one output folder
?
> > >> >> >
> > >> >> > We have not discuss it yet. So, suggestions are welcome!
 :)
> > >> >> >
> > >> >> > Thanks,
> > >> >> > Mikhail
> > >> >> >
> > >> >> >
> > >> >>
> > >> >> Hi Mikhail,
> > >> >>
> > >> >>  OK, great. I'm wondering about how the test classes intended
to
> be
> > >> >> loaded by the bootclasspath loader get distinguished from the
> classes
> > >> >> that are intended to be loaded by the system loader. For the sake
> of
> > >> >> discussion, imagine that all of the test classes under a given
> source
> > >> >> folder (e.g. src/tests/api/java) get compiled to just one output
> > >> folder
> > >> >> (e.g. bin). At test runtime we only want the classes that are
"in
> the
> > >> >> same package as the class under test" to be on the bootclasspath
-
> > >> but
> > >> >> how is this enforced ? If the tests are being run from an Ant
> script
> > >> >> then I think that it is possible to do this using Ant's path-like
> > >> >> structures where wildcards can be specified among the classpath
> and
> > >> >> bootclasspath elements. But how can an equivalent degree of
> > >> separation
> > >> >> be enforced if the tests are being run from an IDE such as Eclipse
> ?
> > >> >>
> > >> >>  I am not exactly an Eclipse "power-user" so it is possible that
I
> am
> > >> >> missing some piece of magic here but it appears to me that if
I
> > >> want to
> > >> >> configure the classpath and bootclasspath for running a particular
> > >> set
> > >> >> of unit tests I can only work at the output folder level. That
is,
> it
> > >> >> can be configured that classes under a root folder of "bin" are
on
> > >> the
> > >> >> classpath and that classes under the root folder "bin-other" are
> > >> >> similarly on the bootclasspath. But there is nothing available
> > >> that only
> > >> >> puts classes in package java.util.* under "bin" on the
> > >> bootclasspath and
> > >> >> everything else on the classpath.
> > >> >>
> > >> >>  All of which makes me wonder if we should aim at compiling
> classes
> > >> >> intended for the bootclasspath to one bin and classes intended
for
> > >> the
> > >> >> classpath to another bin. It wouldn't really make much difference
> > >> to Ant
> > >> >> users but as far as I can tell it would offer more flexibility
to
> IDE
> > >> >> users who would then have the potential to run the tests in a
way
> > >> that
> > >> >> was more familiar to them (runtime configuration, green bar etc).
> > >> >>
> > >> >>
> > >> >> Best regards,
> > >> >> George
> > >> >>
> > >> >> >
> > >> >> >>
> > >> >> >>
> > >> >> >>
> > >> >> >> >> the "bootclasspath" and "classpath" tests distinguished
?
> > >> >> Purely on
> > >> >> >> >> package name ? Did you ever see the append I
wrote to the
> > >> list a
> > >> >> >> couple
> > >> >> >> >> of weeks ago on this topic ? [1]
> > >> >> >> >
> > >> >> >> > Yes, I've seen it. As you could notice we had more
test
> > >> categories.
> > >> >> >> > They are described in the same thread. We might
have
> > >> >> >> > "independent" tests running from bootclasspath and
"specific"
> > >> ones
> > >> >> >> > running from classpath.
> > >> >> >> >
> > >> >> >> >>
> > >> >> >> >> 2) Still in the "Location of the tests in the
directory
> tree"
> > >> >> >> section,
> > >> >> >> >> shouldn't the suggested source folder names
include "java"
> in
> > >> >> there ?
> > >> >> >> >> e.g. <modulename>/src/tests/java/api ?
What is wrong with
> the
> > >> >> >> >> src/test/java (below here is Java code), src/test/resources
> > >> (below
> > >> >> >> here
> > >> >> >> >> is non-code test artefacts) convention ? I notice
that at
> > >> >> present the
> > >> >> >> >> page does not include any mention of where test
resources
> would
> > >> >> go.
> > >> >> >> >
> > >> >> >> > Yes, you are right. It's missing.
> > >> >> >> >
> > >> >> >> > It was supposed to be under test "type", like:
> > >> >> >> > module/src/test/api/java
> > >> >> >> > module/src/test/api/resources
> > >> >> >> >
> > >> >> >> >
> > >> >> >> >>
> > >> >> >> >> 3) What does the sentence "Tests are not separated
by
> > >> >> functionality
> > >> >> >> >> under test, e.g. tests against clone() methods
are not
> > >> >> separated from
> > >> >> >> >> tests against equals() methods" actually mean
?
> > >> >> >> >
> > >> >> >> > That was to address Tim's concern:
> > >> >> >> > "  > 1. Tests are separated on directory level
by 'intention'
> > >> >> >> >
> > >> >> >> > I agree where the intention is broad (i.e. functional
tests
> vs.
> > >> >> >> > performance tests vs. stress tests) but I'm not
convinced
> that
> > >> >> we need
> > >> >> >> > to separate to a precise 'intent', at least I've
never been
> in a
> > >> >> >> > position where I've wished that my my serialization
tests are
> > >> >> separate
> > >> >> >> > from my cloning tests or whatever."
> > >> >> >> >
> > >> >> >> > Feel free to reword if something is not clear in
the proposal
> > >> >> >> >
> > >> >> >> > Thanks,
> > >> >> >> > Mikhail
> > >> >> >> >
> > >> >> >> >
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >> Best regards,
> > >> >> >> >> George
> > >> >> >> >>
> > >> >> >> >> [1]
> > >> >> >> >>
> > >> >> >>
> > >> >>
> > >>
> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200604.mbox/%3c4450A1CC.1060803@googlemail.com%3e
> > >>
> > >> >>
> > >> >> >>
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >> Mikhail Loenko wrote:
> > >> >> >> >> > Hello
> > >> >> >> >> >
> > >> >> >> >> > I would like to make some changes in the
crypto module:
> > >> >> >> >> >
> > >> >> >> >> > - Separate implemetation-independent tests
from
> > >> implementation
> > >> >> >> >> specific
> > >> >> >> >> > ones.
> > >> >> >> >> >
> > >> >> >> >> > - Fix layout according to the proposed
scheme [1]
> > >> >> >> >> >
> > >> >> >> >> > Please let me know if you do any changes
in that module
> then
> > >> >> >> >> > I'll delay restruct.
> > >> >> >> >> >
> > >> >> >> >> > Thanks,
> > >> >> >> >> > Mikhail
> > >> >> >> >> >
> > >> >> >> >> > [1]
> > >> >> >> >> >
> > >> >> >> >>
> > >> >> >>
> > >> >>
> > >>
> http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html
> > >>
> > >> >>
> > >> >> >>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > >>
> > >>
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


-- 
Thanks,
Stepan Mishura
Intel Middleware Products Division

------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message