harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Kleymenov" <kleyme...@gmail.com>
Subject Re: [drlvm][testing] Regression Testing Infrastructure proposal
Date Mon, 04 Dec 2006 12:50:38 GMT

I've made an attempt to resolve the issue with JVM crashes during the
test run. The appropriate patch for DRLVM workspace is provided with
HARMONY-2416 report [1]. This patch extends default functionality of
Ant's <junit> task with JVM crash handling means. The visible benefits
of such an approach are:
    1. There is no need in new test format (java classes with main
function returning predefined result code). So most of the testes can
be implemented in JUnit format.
    2. This approach preserves standard test result reporting
functionality (trough XML to HTML reports) and there is no need to
implement it as in case of using of new test format.
    3. It's easy to switch between standard and enhanced test
execution. The <junit> task redefining can be made dependable on some

To try the patch, just apply it on the built DRLVM workspace, than
make "ant update", and then "ant reg.test". The test results in html
format will be placed in semis/reg.tests/reports/html directory under
the DRLVM's build directory. Note that H000 regression test is failed
because of DRLVM crash.

Please, try the patch and give your thoughts and advices about it.

So with this patch the current status of regression testing
infrastructure is as follows:
    1. There are the working means for automated regression testing
and report generation.
    2. The test format for regression test is JUnit. The sources for
regression tests are placed in src/test/regression/Hxyz directories,
where Hxyz is named according to corresponding JIRA report.
    3. There are 4 regression tests included in the regression test suite.
    4. I'm working on automated test integration (don/t list each
particular test in reg.test.xml file, but search them automatically).
    5. I'm working on document describing directions for developers of
new regression tests.
    6. Existing regression tests under src/test directory will be
integrated to regression test suite.
    7. It is ready to be used by cruise control system.

Thank you,
Alexander Kleymenov

[1] http://issues.apache.org/jira/browse/HARMONY-2416

View raw message