harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stepan Mishura" <stepan.mish...@gmail.com>
Subject Re: Excluded tests in x-net module or back to bootclasspath issue
Date Wed, 15 Mar 2006 09:29:05 GMT
Hi Tim,

On 3/14/06, Tim Ellison <t.p.ellison@gmail.com> wrote:
>
> I agree that they need to be reinstated Stepan, I'd just like to
> separate out the suite of tests that are pure API tests (and therefore
> expected to run on all compliant implementations, modulo their known
> bugs/departures) and the internal implementation tests that we would
> only expect to pass on Harmony code.
>
> I had hoped that we could separate them using JUnit test suites, but as
> Geir pointed out the Ant support for JUnit suites produces very limited
> reports.  I'd always run suites directly from Eclipse and could see all
> the required info.
>
> So it seems that the options are to (a) figure out why Ant is hiding the
> data, and fix that; or (b) live with it and invent a naming scheme for
> Harmony implementation tests that keep them separate from the API tests.


I'd suggest the following tests separation:
<MODULE_ROOT>/src/test/java
    (java|javax|org)/**/*Test.java (excluding org/apache/harmony/tests)
     org/apache/harmony/tests/**/*Test.java

First folder contains tests for 'package access' functionality and tested
classes must be on bootclasspath. Second folder keeps tests for public API (
i.e. 'compliant' tests). To avoid test code duplication the preferred folder
for placing new tests is the second one - if a test doesn't depend on
package access functionality them it goes to 'compliant' tests. So it is up
to a developer to decide where to place a test. Also it may be makes sense
to compile tests to different 'bin' folders.

So we'll have the next 'testing modes':
1) Compliant mode: 'compliant' tests are run on some java certified
implementation.
2) Cross-compliant mode: all tests run against the following JRE
configuration - a module's jar + compliant java implementation (i.e. a
module's jar and 'package access' tests are on the bootclasspath)
3) Self-testing mode: all tests run against Harmony implementation


As (I think it was) Mikhail pointed out a while ago, using class naming
> schemes is far from ideal, as there may be a significant repetition of
> test cases and the suites they fit into.



I believe that the separation suggested above may solve this issue.

 If we want to support testing
> with 'exotic' configurations then we will need some 'suites' that assume
> basic set-up, and others that can assume a test http server, ldap
> server, database, and so on.


I'd prefer to avoid external dependencies for unit testing as much as
possible. Otherwise specific (some special set-up is required) tests should
be placed to a separate test suite.

Thanks,
Stepan.

So my preference is to figure out why Ant produces its limited report
> for JUnit suites.
>
> Matt: can you shed any light on this?
>
> Regards,
> Tim
>
> Stepan Mishura wrote:
> > Hi Tim,
> >
> > Currently you excluded 12 tests in 'x-net' module. The ant script file
> > contains the following comments for them: "The following tests are
> excluded
> > because they fail for reasons to be determined"
> > The reason is well known and it was discussed a lot. These tests depend
> on
> > package access functionality and BC provider jar is still required to be
> on
> > bootclasspath (I don't remember that we solved this issue). I know that
> you
> > are allergic to putting all stuff on bootclasspath but I'd prefer keep
> them
> > running to track regressions rather then adding them to exclude list and
> > forgetting about them for a long time.
> >
> > Thanks,
> > Stepan Mishura
> > Intel Middleware Products Division
> >
>
> --
>
> Tim Ellison (t.p.ellison@gmail.com)
> IBM Java technology centre, UK.
>



--
Thanks,
Stepan Mishura
Intel Middleware Products Division

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message