harmony-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From var...@apache.org
Subject svn commit: r489018 - in /harmony/enhanced/drlvm/trunk: build/make/build.xml build/make/targets/reg.test.run.xml build/make/targets/reg.test.xml src/test/regression/H1694/H1694.java src/test/regression/README.txt
Date Wed, 20 Dec 2006 10:30:57 GMT
Author: varlax
Date: Wed Dec 20 02:30:56 2006
New Revision: 489018

URL: http://svn.apache.org/viewvc?view=rev&rev=489018
Log:
HARMONY-2540 [drlvm][testing] Regression Testing Infrastructure implementation.
Applied the original patch and simplified it a bit;
also reduced coupling with the drlvm build system.
Tested on WinXP and SUSE9

Added:
    harmony/enhanced/drlvm/trunk/src/test/regression/README.txt   (with props)
Modified:
    harmony/enhanced/drlvm/trunk/build/make/build.xml
    harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.run.xml
    harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.xml
    harmony/enhanced/drlvm/trunk/src/test/regression/H1694/H1694.java

Modified: harmony/enhanced/drlvm/trunk/build/make/build.xml
URL: http://svn.apache.org/viewvc/harmony/enhanced/drlvm/trunk/build/make/build.xml?view=diff&rev=489018&r1=489017&r2=489018
==============================================================================
--- harmony/enhanced/drlvm/trunk/build/make/build.xml (original)
+++ harmony/enhanced/drlvm/trunk/build/make/build.xml Wed Dec 20 02:30:56 2006
@@ -421,12 +421,8 @@
     </target>
 
     <!-- Main target to run regression test /-->
-    <target name="reg.test" depends="setup, init">
-        <property name="target" value="reg.test" />
-        <subant buildpath="." antfile="build_component.xml" target="build" inheritall="true"
failonerror="true">
-            <property name="_component" value="vm" />
-            <property name="_target" value="${target}" />
-        </subant>
+    <target name="reg.test" depends="init">
+        <ant antfile="targets/reg.test.xml" target="reg.test"/>
     </target>
 
     <!-- process (build/create javadoc/etc...) all the specified components -->

Modified: harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.run.xml
URL: http://svn.apache.org/viewvc/harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.run.xml?view=diff&rev=489018&r1=489017&r2=489018
==============================================================================
--- harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.run.xml (original)
+++ harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.run.xml Wed Dec 20 02:30:56 2006
@@ -15,39 +15,127 @@
      See the License for the specific language governing permissions and
      limitations under the License. -->
 
-<project name="REGRESSION TESTS RUN">
-    <property name="reg.test.timeout" value="180000" />
-    <property name="jvm.under.test" value="${build.deploy.dir}/jdk/jre/bin/java"/>
-
-    <target name="run-reg-tests">
-        <echo>
-==================================
-     Run regression tests
-==================================
-        </echo>
-    
-        <!-- Execution of a test in JUnit format -->
-        <run-junit-test test="org.apache.harmony.drlvm.tests.regression.h1800.ExecTest"/>
-
-        <!-- Execution of a test in JUnit format -->
-        <run-junit-test test="org.apache.harmony.drlvm.tests.regression.h788.Test"/>
-
-        <!-- Execution of a java test with main function and expected result code -->
-        <run-pjava-test test="org.apache.harmony.drlvm.tests.regression.h1694.H1694" expected-code="104"/>
-        
-        <!-- Launch the tests with a custom launchers placed near the tests -->
-        <run-custom-launcher launcher="${reg.test.src}/H2151/run.test.xml"/>
-        <run-custom-launcher launcher="${reg.test.src}/H0000/run.test.xml"/>
-
-    </target>
-
-    <!-- Macrodefinition launching the custom launcher -->
-    <macrodef name="run-custom-launcher">
-        <attribute name="launcher"/>
-        <sequential>
-            <ant antfile="@{launcher}" target="run-test" inheritAll="true" inheritRefs="true"/>
-        </sequential>
-    </macrodef>
+ <project name="REGRESSION TESTS RUN" default="do-regression-testing">
+     
+     <property name="jvm.under.test" value="${build.deploy.dir}/jdk/jre/bin/java"/>
+     <property name="reg.test.timeout" value="180000" />
+     <property name="reg.test.halt.on.error" value="false"/>
+     <!-- Mark used by custom test launchers for test failure notification -->
+     <property name="reg.test.failed.mark" value="-=@@@ Reg Test Failed @@@=-"/>
+  
+     <target name="do-regression-testing" 
+             depends="launch-tests,make-report,check-results">
+     </target>
+ 
+     <target name="launch-tests">
+          <echo>
+  ==================================
+       Run regression tests
+  ==================================
+          </echo>
+  
+         <for param="reg.test.2launch.src.dir">
+             <!-- Go through directories containing regression tests -->
+             <dirset dir="${reg.test.src}">
+                 <include name="H*"/>
+             </dirset>
+             <sequential>
+                 <!-- Set up the property pointing to the natives path -->
+                 <var name="reg.test.2launch.native.path" unset="true"/>
+                 <pathconvert property="reg.test.2launch.native.path">
+                     <path path="@{reg.test.2launch.src.dir}"/>
+                     <map from="${reg.test.src}" to="${reg.test.native.path}"/>
+                 </pathconvert>
+                 <if>
+                     <available file="@{reg.test.2launch.src.dir}/run.test.xml"/>
+                     <then>
+                         <!-- Launch the tests with a custom launchers 
+                                                 placed near the tests -->
+                         <run-custom-launcher 
+                             launcher="@{reg.test.2launch.src.dir}/run.test.xml"/>
+                     </then>
+                     <else>
+                         <!-- Execution of unit tests checking for regression -->
+                         <for param="reg.unit.test.src">
+                             <!-- Go through unit tests in the directory -->
+                             <fileset dir="@{reg.test.2launch.src.dir}">
+                                 <include name="*Test.java"/>
+                             </fileset>
+                             <sequential>
+                                 <propertyregex override="yes"
+                                   property="unit.test.class" 
+                                   input="@{reg.unit.test.src}"
+                                   regexp=".*H(\d\d*)[/\\](..*)\.java" 
+                                   replace="${reg.test.package}.h\1.\2"/>
+                                 <run-junit-test test="${unit.test.class}"/>
+                             </sequential>
+                         </for>
+                     </else>
+                 </if>
+             </sequential>
+         </for>
+  
+     </target>
+ 
+     <target name="make-report">
+          <junitreport todir="${reg.test.dir}/reports">
+              <fileset dir="${reg.test.dir}/reports">
+                  <include name="TEST*-*.xml"/>
+              </fileset>
+              <report format="frames" todir="${reg.test.dir}/reports/html"/>
+          </junitreport>
+         <echo message="========================================"/>
+         <echo message="Regression testing report was stored in:"/>
+         <echo message="========================================"/>
+         <echo message="${reg.test.dir}/reports/html"/>
+         <echo message="========================================"/>
+     </target>
+     
+     <target name="check-results">
+         <fail if="reg.test.errors" 
+             message="There were test failures"/>
+     </target>
+  
+      <!-- Macrodefinition launching the custom launcher -->
+      <macrodef name="run-custom-launcher">
+          <attribute name="launcher"/>
+          <sequential>
+             <!-- 
+                 Set up try-catch block to catch the test failure.
+ 
+                 It's possible to use <antfetch> task for this purpose:
+                     <antfetch 
+                         antfile="@{launcher}" target="run-test" 
+                         return="reg.test.errors" 
+                         inheritAll="true" inheritRefs="true"/>
+                 but it fails with NPE while <macrodef> processing...
+             -->
+             <trycatch property="reg.test.failure.message"
+                       reference="reg.test.failure.reference">
+                 <try>
+                     <ant antfile="@{launcher}" target="run-test" 
+                              inheritAll="true" inheritRefs="true">
+                          <property name="reg.test.custom.launched" value="true"/>
+                     </ant>
+                 </try>
+                 <catch>
+                     <if>
+                         <!-- it was a test failure -->
+                         <contains string="${reg.test.failure.message}" 
+                                   substring="${reg.test.failed.mark}"/>
+                         <then>
+                             <!-- set up the property -->
+                             <property name="reg.test.errors" value="true"/>
+                         </then>
+                         <else>
+                             <!-- propagate the error -->
+                             <throw refid="reg.test.failure.reference"/>
+                         </else>
+                     </if>
+                 </catch>
+             </trycatch>
+          </sequential>
+      </macrodef>
 
     <!-- Macrodefinition launching JUnit test -->
     <macrodef name="run-junit-test">
@@ -57,11 +145,11 @@
         <sequential>
             <junit jvm="${jvm.under.test}"
                    fork="yes"
-                   haltonfailure="true"
-                   haltonerror="true"
+                   haltonfailure="false"
+                   haltonerror="false"
                    printsummary="withOutAndErr"
                    dir="${basedir}"
-                   failureproperty="@{test}.failed"
+                   failureproperty="reg.test.errors"
                    timeout="${reg.test.timeout}">
                 <test name="@{test}" todir="${reg.test.dir}/reports"/>
                 <classpath>
@@ -69,10 +157,11 @@
                 </classpath>
                 <junit-element/>
                 <formatter type="brief"/>
+                <formatter type="xml"/>
             </junit>
         </sequential>
     </macrodef>
-
+     
     <!-- Macrodefinition launching JVMTI JUnit test -->
     <macrodef name="run-jvmti-test">
         <attribute name="test"/>
@@ -82,16 +171,9 @@
             <run-junit-test test="@{test}" vmarg="@{vmarg}">
                 <junit-element>
                     <jvmarg value="-Dvm.assert_dialog=0" />
-                    <select os="win">
-                        <jvmarg
-                        value="-agentpath:${reg.test.native.path}/@{agent}.dll"
-                        />
-                    </select>
-                    <select os="lnx">
                         <jvmarg
-                        value="-agentpath:${reg.test.native.path}/lib@{agent}.so"
+                        value="-agentpath:${reg.test.2launch.native.path}/${lib.prefix}@{agent}${lib.suffix}"
                         />
-                    </select>
                 </junit-element>
             </run-junit-test>
         </sequential>

Modified: harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.xml
URL: http://svn.apache.org/viewvc/harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.xml?view=diff&rev=489018&r1=489017&r2=489018
==============================================================================
--- harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.xml (original)
+++ harmony/enhanced/drlvm/trunk/build/make/targets/reg.test.xml Wed Dec 20 02:30:56 2006
@@ -16,15 +16,38 @@
      limitations under the License. -->
 
 <project name="REGRESSION TESTS">
+    
+    <import file="${basedir}/targets/reg.test.run.xml"/>
+    
     <property name="reg.test.src" location="${basedir}/../../src/test/regression"/>
     <property name="reg.test.dir" location="${build.semi.dir}/reg.tests"/>
     <property name="reg.test.class.path" location="${reg.test.dir}/classes"/>
     <property name="reg.test.native.path" location="${reg.test.dir}/native"/>
-    <!-- TODO: check the existence of jasmin.jar -->
     <property name="jasmin.jar" location="./tmp/jasmin.jar"/>
-    <property name="reg.test.package" value="org/apache/harmony/drlvm/tests/regression"/>
+    <property name="reg.test.package" value="org.apache.harmony.drlvm.tests.regression"/>
+    
+    <propertyregex property="reg.test.unpackage" 
+        input="${reg.test.package}" regexp="\." replace="/"/>
+
+    <if> <!-- Set up os specific settings - library suffix and prefix -->
+        <os family="windows"/>
+        <then>
+            <property name="lib.prefix" value=""/>
+            <property name="lib.suffix" value=".dll"/>
+        </then>
+        <elseif>
+            <os family="unix"/>
+            <then>
+                <property name="lib.prefix" value="lib"/>
+                <property name="lib.suffix" value=".so"/>
+            </then>
+        </elseif>
+        <else>
+            <fail>Is not designed for this platform!</fail>
+        </else>
+    </if>
 
-    <target name="compile-reg-tests-prepare" depends="init_component">
+    <target name="compile-reg-tests-prepare" depends="">
         <mkdir dir="${reg.test.class.path}" />
         <mkdir dir="${reg.test.native.path}" />
 
@@ -55,7 +78,7 @@
                         </filtermapper>
                         <regexpmapper from="^(?:.*/)?H(\d\d*)/([^/].*).j$$" 
                                       to="h\1/\2.class"/>
-                        <globmapper from="*" to="${reg.test.package}/*"/>
+                        <globmapper from="*" to="${reg.test.unpackage}/*"/>
                     </chainedmapper>
                 </mapper>
             </depend>
@@ -88,83 +111,85 @@
                         <regexpmapper 
                             from="^(?:.*/)?H(\d\d*)/([^/].*).java$$"
                             to="h\1/\2.class"/>
-                        <globmapper from="*" to="${reg.test.package}/*"/>
+                        <globmapper from="*" to="${reg.test.unpackage}/*"/>
                     </chainedmapper>
                 </mapper>
             </depend>
         </javac>
     </target>
 
-    <target name="compile-reg-tests-native"
-        depends="compile-reg-tests-prepare, init_component">
-
-        <pathconvert property="reg.test.src.native.list"
-                     refid="reg.test.src.native.set"
-                     pathsep=","/>
-        <mkdir dir="${reg.test.native.path}/_obj"/>
+    <target name="compile-reg-tests-native" depends="compile-reg-tests-prepare">
 
-        <for list="${reg.test.src.native.list}" param="csource"
-             delimiter=",">
+        <for param="nativefile">
+            <path>
+                <fileset dir="${reg.test.src}" includes="**/*.c,**/*.cpp">
+                    <depend targetdir="${reg.test.native.path}">
+                        <mapper>
+                        <chainedmapper>
+                            <filtermapper>
+                                <replacestring from="\" to="/"/>
+                            </filtermapper>
+                            <regexpmapper from="(H\d\d*)/([^/]+)\.c.?.?$$" 
+                                          to="\1/${lib.prefix}\2${lib.suffix}"/>
+                        </chainedmapper>
+                        </mapper>
+                    </depend>
+                </fileset>
+            </path>
             <sequential>
-                <var name="outname" unset="true" />
-                <!-- basename property="outname" file="@{csource}" suffix=".cpp"/ -->
-                <pathconvert property="outname">
-                    <path path="@{csource}"/>
+                <var name="outfile" unset="true"/>
+                <pathconvert property="outfile">
+                    <path path="@{nativefile}"/>
                     <chainedmapper>
-                        <flattenmapper/>
-                        <regexpmapper from="^(..*)\.c.?.?$$" to="\1"/>
+                        <filtermapper>
+                            <replacestring from="\" to="/"/>
+                        </filtermapper>
+                        <regexpmapper from="/(H\d\d*)/([^/]+)\.c.?.?$$" to="\1/\2"/>
                     </chainedmapper>
                 </pathconvert>
+                <var name="outdir" unset="true"/>
+                <dirname property="outdir" file="${reg.test.native.path}/${outfile}"/>
+                <!-- mkdir dir="${outdir}"/ -->
+                <var name="objdir" unset="true"/>
+                <property name="objdir" value="${outdir}/_obj"/>
+                <mkdir dir="${objdir}"/>
 
-                <echo message="## Compiling reg native ${outname} in: ${reg.test.native.path}"
/>
+                <echo message="## Compiling reg native ${outname}" />
                 <cc name="${build.cxx}"
                     debug="true"
                     runtime="dynamic"
                     multithreaded="true"
                     link="shared"
-                    outfile="${reg.test.native.path}/${outname}"
-                    objdir="${reg.test.native.path}/_obj">
-                    <includepath path="${build.deploy.dir}jdk/jre/include" />
-                    <select os="lnx">
+                    outfile="${reg.test.native.path}/${outfile}"
+                    objdir="${objdir}">
+                    <includepath path="${build.deploy.dir}/jdk/include" />
+                    <!--select os="lnx">
                         <linkerarg value="-lstdc++" />
-                    </select>
-                    <fileset file="@{csource}"/>
+                    </select-->
+                    <fileset file="@{nativefile}"/>
                 </cc>
             </sequential>
         </for>
     </target>
 
     <target name="pre-reg-test" depends="compile-reg-tests-java, compile-reg-tests-native">
-    <!--target name="pre-reg-test" depends="compile-reg-tests-native"-->
-    <!-- target name="pre-reg-test" depends="init_component"-->
+        <delete quiet="true" dir="${reg.test.dir}/reports" />
+        <mkdir dir="${reg.test.dir}/reports" />       
+
         <condition property="junit.jar" value="${junit.home}/junit.jar">
             <isset property="junit.home" />
         </condition>
-        <property name="junit.jar"
-                  location="${external.dep.CLASSLIB}/depends/jars/junit_3.8.2/junit.jar"
/>
-        <if>
-            <not>
-                <available file="${junit.jar}"/>
-            </not>
-            <then>
-                <echo>
+        <property name="junit.jar" 
+            location="${external.dep.CLASSLIB}/depends/jars/junit_3.8.2/junit.jar"/>
+        <available file="${junit.jar}" property="junit.found"/>
+        <fail unless="junit.found">
                 ==============================================
                 Please set the classpath of junit as follows:
                 build.bat -Djunit.jar=%JUNIT_HOME% test
                 ==============================================
-                </echo>
-                <fail message="reg tests misconfigured" />
-            </then>
-        </if>
-        <sequential>
-            <delete quiet="true" dir="${reg.test.dir}/reports" />
-            <delete quiet="true" dir="${reg.test.dir}/ref-reports" />
-            <mkdir dir="${reg.test.dir}/reports" />
-            <mkdir dir="${reg.test.dir}/ref-reports" />
-        </sequential>
+        </fail>
     </target>
 
-    <target name="reg.test" depends="pre-reg-test" >
-        <ant antfile="${ant.file.REGRESSION TESTS RUN}" target="run-reg-tests" inheritAll="true"
inheritRefs="true"/>
-    </target>
+    <target name="reg.test" depends="pre-reg-test, do-regression-testing" />
+    
 </project>

Modified: harmony/enhanced/drlvm/trunk/src/test/regression/H1694/H1694.java
URL: http://svn.apache.org/viewvc/harmony/enhanced/drlvm/trunk/src/test/regression/H1694/H1694.java?view=diff&rev=489018&r1=489017&r2=489018
==============================================================================
--- harmony/enhanced/drlvm/trunk/src/test/regression/H1694/H1694.java (original)
+++ harmony/enhanced/drlvm/trunk/src/test/regression/H1694/H1694.java Wed Dec 20 02:30:56
2006
@@ -1,7 +1,10 @@
 package org.apache.harmony.drlvm.tests.regression.h1694;
 
-class H1694 {
-    public int test() {
+import junit.framework.TestCase;
+
+public class H1694 extends TestCase {
+
+    public void test() {
         Object arrayOfObjects[] = new Object[10000]; // array of objects
 
         // padding memory
@@ -18,11 +21,6 @@
         }
 
         System.out.println("Test passed");
-        return 104; // return pass
-    }
- 
-    public static void main(String[] args) {
-        System.exit(new H1694().test());
     }
 }
  

Added: harmony/enhanced/drlvm/trunk/src/test/regression/README.txt
URL: http://svn.apache.org/viewvc/harmony/enhanced/drlvm/trunk/src/test/regression/README.txt?view=auto&rev=489018
==============================================================================
--- harmony/enhanced/drlvm/trunk/src/test/regression/README.txt (added)
+++ harmony/enhanced/drlvm/trunk/src/test/regression/README.txt Wed Dec 20 02:30:56 2006
@@ -0,0 +1,94 @@
+               Directions for Regression Test Developers
+               =========================================
+
+    This file describes conventions accepted for VM regression test suite
+    and steps needed to integrate new tests into the suite.
+    ---------------------------------------------------------------------
+
+
+Test Format and Test Naming conventions
+---------------------------------------
+
+ 1. The tests are supposed to be in JUnit test format
+
+ 2. The source files of each regression test should be placed in a directory
+    named H{NUMBER}, where {NUMBER} (here and below) is an ID
+    of JIRA issue the test is related to.
+        For example: H1234
+
+ 3. The name of the main class should end with Test suffix
+        For example: SomethingTest.java
+
+ 4. The package of the test should be
+    org.apache.harmony.drlvm.tests.regression.h{NUMBER}
+       For example: org.apache.harmony.drlvm.tests.regression.h1234
+
+ 5. The source files for the tests can be in Java, Java Assembler, and
+    C/C++ programming languages.
+
+
+New Test Integration
+--------------------
+
+To integrate new test in the regression test suite:
+
+ 1. Make sure it complies with conventions described above.
+
+ 2. Put the test sources into H{NUMBER} directory.
+
+ 3. By default test execution entry point is a class ending with Test
+    suffix. It will be automatically launched by regression test
+    infrastructure as a JUnit test. If there are several classes ending
+    with Test suffix all of them will be launched.
+
+After that regression test infrastructure will be able to compile and
+execute your test.
+
+
+Custom Test Launchers
+---------------------
+
+If there is a need in some more sophisticated test launching mechanism
+(JVM parameters or environment variables need to be specified in order to
+reproduce the problem, etc.) it can be done by using CUSTOM LAUNCHER.
+It is an Ant build file named run.test.xml and placed near the test
+sources. The target named "run-test" should specify the way in which the
+test is launched. This Ant's file is executed in context of DRLVM build
+system and inherits all the properties, references, and macrodefinitions
+defined there. The custom launcher is started from
+            build/make/targets/reg.test.run.xml
+file which contains some useful macrodefinitions to simplify custom
+launchers.
+
+ Note:  If there are tests with Test suffix along with the custom launcher
+ -----  they won't be automatically launched by regression testing
+        infrastructure.
+
+
+Some useful properties for custom launchers:
+-------------------------------------------
+
+    ${reg.test.class.path} - the path to where java and
+        java assembler sources were compiled
+
+    ${reg.test.2launch.native.path} - the path to where native
+        sources were compiled
+
+
+Examples
+--------
+The best way to learn something is to use examples. Please, use existing 
+regression tests for this purpose. The following tests can act as a starting 
+points:
+
+    H1694 - Simple JUnit test automatically compiled and 
+            launched by regression testing framework.
+
+    H2151 - The test consisting of java and native code.
+            It uses custom launcher for execution.
+
+    H788  - The test consisting of java and java assembler
+            sources. Source files are automatically compiled by
+            regression testing framework. The entry point for the test
+            is JUnit test implemented by Test.java source file.
+

Propchange: harmony/enhanced/drlvm/trunk/src/test/regression/README.txt
------------------------------------------------------------------------------
    svn:eol-style = native



Mime
View raw message