harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: luni test script tweaks
Date Tue, 07 Mar 2006 22:11:09 GMT
Geir Magnusson Jr wrote:
> 
> 
> Tim Ellison wrote:
>> Geir Magnusson Jr wrote:
>>>
>>> Tim Ellison wrote:
>>>> Geir Magnusson Jr wrote:
>>>>> do you want to discuss what you are undoing?
>>>> I assume the monster diff is due to EOL differences or something...
>>>>
>>>> 1. I've make the test report location a variable
>>> I thought you pointed it back into luni?
>>
>> Nope, see  http://svn.apache.org/viewcvs?rev=383948&view=rev
>>
>>>> 2. I've undone somebody else's undoing of the test suite invocation
>>> Right, because we didn't get the same info from doing the suite as we
>>> did form running individual tests, IIRC.
>>
>> Such as...?  Launching the VM for every individual test case is not
>> helpful here.
> 
> forkmode="once"

Yep

>>> Since the suite was only a set of tests (no extra setup)
>>
>> what?
> 
> IOW, you don't do anything else.  It's just a bag.

Once we get HARMONY-57 tests installed we will wrap the suite in the
exclusions list support code, and can apply any other logic to select
which tests are run, check for performance regressions, etc.  It will
still look like one JUnit suite to the Ant task.  If we build the suite
from <batchtest> including *Test.java then I don't think we can do that?

>>> and explicitly for Eclipse users because
>>> Eclipse's JUnit support is/was broken ;)
>>
>> I use Eclipse all the time (and use the suite all the time from
>> Eclipse).  This tweak makes the Ant script behave that way too.
> 
> I thought you had to use the suite because it wouldn't run more than one
> test at a time... maybe that was before the current version...

As above, running a decorated test suite from Eclipse works ok.

>>> I thought there was no harm in
>>> running the individual tests.  It also makes it easy to just drop in a
>>> test...
>>
>> I've had no problem so far adding tests.  There is value in having
>> multiple suites (which we have only eluded to so far on the list).
> 
> Sure - but that would be for developer testing, right?  Meaning we'd
> want to organize into sensible suites for subareas of the library?

We can also use multiple suites for different 'exotic' configurations as
described by Mikhail a while ago.  That is, we may choose to reserve
some tests for environments that are complex to set-up and only run them
as a specific test target (rather then ant -f build-tests.xml).

> The thing that we lose is individual test information in reporting...
> that's why I changed the ant script to just run the tests, and ignore
> the suites, as I thought of that as a crutch...

Just looking at a report from a test suite, I can see the individual
tests that were run.  Is that what you meant?

Here's what I did:

cd make
ant   (to build the classlib code)
copy across the VM into deploy, and XML jars into lib/boot
ant -f build-test.xml test-luni,gen-report

then browse ..\build\test_report\html\index.html


Regards,
Tim

>>>>> tellison@apache.org wrote:
>>>>>> Author: tellison
>>>>>> Date: Tue Mar  7 10:08:47 2006
>>>>>> New Revision: 383950
>>>>>>
>>>>>> URL: http://svn.apache.org/viewcvs?rev=383950&view=rev
>>>>>> Log:
>>>>>> Use the test suite, and put the results in the reporting dir
>>>>>>
>>>>>> Modified:
>>>>>>  
>>>>>> incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Modified:
>>>>>> incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml
>>>>>>
>>>>>>
>>>>>>
>>>>>> URL:
>>>>>> http://svn.apache.org/viewcvs/incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml?rev=383950&r1=383949&r2=383950&view=diff
>>>>>>
>>>>>>
>>>>>>
>>>>>> ==============================================================================
>>>>>>
>>>>>>
>>>>>>
>>>>>> ---
>>>>>> incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml
>>>>>>
>>>>>>
>>>>>> (original)
>>>>>> +++
>>>>>> incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml
>>>>>>
>>>>>>
>>>>>> Tue Mar  7 10:08:47 2006
>>>>>> @@ -1,111 +1,99 @@
>>>>>> -<?xml version="1.0" encoding="UTF-8"?>
>>>>>> -<!--
>>>>>> -    Copyright 2006 The Apache Software Foundation or its licensors,
>>>>>> as applicable.
>>>>>> -  -    Licensed under the Apache License, Version 2.0 (the
>>>>>> "License");
>>>>>> -    you may not use this file except in compliance with the License.
>>>>>> -    You may obtain a copy of the License at
>>>>>> -  -       http://www.apache.org/licenses/LICENSE-2.0
>>>>>> -  -    Unless required by applicable law or agreed to in writing,
>>>>>> software
>>>>>> -    distributed under the License is distributed on an "AS IS"
>>>>>> BASIS,
>>>>>> -    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
or
>>>>>> implied.
>>>>>> -    See the License for the specific language governing permissions
>>>>>> and
>>>>>> -    limitations under the License.
>>>>>> --->
>>>>>> -
>>>>>> -<project name="Common_LUNI_Build">
>>>>>> -   -    <target name="compile.java" description="Compile LUNI
java
>>>>>> code">
>>>>>> -        <echo message="Compiling LUNI classes from
>>>>>> ${hy.luni.src.main.java}" />
>>>>>> -       -        <mkdir dir="${hy.luni.bin.main}" />
>>>>>> -
>>>>>> -        <javac sourcepath=""
>>>>>> -            srcdir="${hy.luni.src.main.java}"
>>>>>> -            destdir="${hy.luni.bin.main}"
>>>>>> -            source="${source.ver}"
>>>>>> -            debug="${java.debug.option}">
>>>>>> -
>>>>>> -            <bootclasspath>
>>>>>> -                <fileset dir="${hy.target}/jre/lib/boot">
>>>>>> -                    <include name="*.jar" />
>>>>>> -                </fileset>
>>>>>> -            </bootclasspath>
>>>>>> -        </javac>
>>>>>> -    </target>
>>>>>> -   -    <target name="build.jar">
>>>>>> -        <jar destfile="${hy.target}/jre/lib/boot/luni.jar"
>>>>>> manifest="${hy.luni}/META-INF/MANIFEST.MF">
>>>>>> -            <fileset dir="${hy.luni.bin.main}" />
>>>>>> -        </jar>
>>>>>> -    </target>
>>>>>> -
>>>>>> -   -    <target name="compile.tests">
>>>>>> -        <echo message="Compiling LUNI tests from
>>>>>> ${hy.luni.src.test.java}" />
>>>>>> -
>>>>>> -        <mkdir dir="${hy.luni.bin.test}" />
>>>>>> -
>>>>>> -        <javac srcdir="${hy.luni.src.test.java}"
>>>>>> -               destdir="${hy.luni.bin.test}"
>>>>>> -               sourcepath=""
>>>>>> -               source="${source.ver}"
>>>>>> -               debug="${java.debug.option}">
>>>>>> -
>>>>>> -            <bootclasspath>
>>>>>> -                <fileset dir="${hy.target}/jre/lib/boot">
>>>>>> -                    <include name="*.jar" />
>>>>>> -                </fileset>
>>>>>> -            </bootclasspath>
>>>>>> -        </javac>
>>>>>> -    </target>
>>>>>> -
>>>>>> -
>>>>>> -    <target name="run.tests">
>>>>>> -       -        <property name="test.dir"
>>>>>> value="../../../../target/test_report" />
>>>>>> -        <mkdir dir="${test.dir}" />
>>>>>> -
>>>>>> -        <junit fork="yes"
>>>>>> -            forkmode="once"
>>>>>> -            printsummary="withOutAndErr"
>>>>>> -            errorproperty="test.error"
>>>>>> -            showoutput="on"
>>>>>> -            dir="${hy.luni.bin.test}"
>>>>>> -            jvm="${hy.target}/jre/bin/java">
>>>>>> -
>>>>>> -            <jvmarg value="-showversion"/>
>>>>>> -
>>>>>> -            <env key="JAVA_HOME" value=""/>
>>>>>> -
>>>>>> -            <classpath>
>>>>>> -                <pathelement path="${hy.luni.bin.test}"/>
>>>>>> -            </classpath>
>>>>>> -
>>>>>> -            <!--
>>>>>> -            <test name="org.apache.harmony.tests.luni.AllTests"
>>>>>> -                haltonfailure="no"
>>>>>> -                todir="${test.dir}">
>>>>>> -                <formatter type="xml" />
>>>>>> -            </test>
>>>>>> -            -->
>>>>>> -           -            <formatter type="xml" />
>>>>>> -            <batchtest todir="${test.dir}">
>>>>>> -                <fileset dir="../../src/test/java">
>>>>>> -                    <include name="**/*Test.java"/>
>>>>>> -                </fileset>
>>>>>> -            </batchtest>
>>>>>> -
>>>>>> -        </junit>
>>>>>> -       -    </target>
>>>>>> -   -   -    <target name="copy.resources">
>>>>>> -        <!-- Nothing for LUNI -->
>>>>>> -    </target>
>>>>>> -</project>
>>>>>> -
>>>>>> +<?xml version="1.0" encoding="UTF-8"?>
>>>>>> +<!--
>>>>>> +    Copyright 2006 The Apache Software Foundation or its licensors,
>>>>>> as applicable.
>>>>>> +  +    Licensed under the Apache License, Version 2.0 (the
>>>>>> "License");
>>>>>> +    you may not use this file except in compliance with the License.
>>>>>> +    You may obtain a copy of the License at
>>>>>> +  +       http://www.apache.org/licenses/LICENSE-2.0
>>>>>> +  +    Unless required by applicable law or agreed to in writing,
>>>>>> software
>>>>>> +    distributed under the License is distributed on an "AS IS"
>>>>>> BASIS,
>>>>>> +    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
or
>>>>>> implied.
>>>>>> +    See the License for the specific language governing permissions
>>>>>> and
>>>>>> +    limitations under the License.
>>>>>> +-->
>>>>>> +
>>>>>> +<project name="Common_LUNI_Build">
>>>>>> +   +    <target name="compile.java" description="Compile LUNI
java
>>>>>> code">
>>>>>> +        <echo message="Compiling LUNI classes from
>>>>>> ${hy.luni.src.main.java}" />
>>>>>> +       +        <mkdir dir="${hy.luni.bin.main}" />
>>>>>> +
>>>>>> +        <javac sourcepath=""
>>>>>> +            srcdir="${hy.luni.src.main.java}"
>>>>>> +            destdir="${hy.luni.bin.main}"
>>>>>> +            source="${source.ver}"
>>>>>> +            debug="${java.debug.option}">
>>>>>> +
>>>>>> +            <bootclasspath>
>>>>>> +                <fileset dir="${hy.target}/jre/lib/boot">
>>>>>> +                    <include name="*.jar" />
>>>>>> +                </fileset>
>>>>>> +            </bootclasspath>
>>>>>> +        </javac>
>>>>>> +    </target>
>>>>>> +   +    <target name="build.jar">
>>>>>> +        <jar destfile="${hy.target}/jre/lib/boot/luni.jar"
>>>>>> manifest="${hy.luni}/META-INF/MANIFEST.MF">
>>>>>> +            <fileset dir="${hy.luni.bin.main}" />
>>>>>> +        </jar>
>>>>>> +    </target>
>>>>>> +
>>>>>> +   +    <target name="compile.tests">
>>>>>> +        <echo message="Compiling LUNI tests from
>>>>>> ${hy.luni.src.test.java}" />
>>>>>> +
>>>>>> +        <mkdir dir="${hy.luni.bin.test}" />
>>>>>> +
>>>>>> +        <javac srcdir="${hy.luni.src.test.java}"
>>>>>> +               destdir="${hy.luni.bin.test}"
>>>>>> +               sourcepath=""
>>>>>> +               source="${source.ver}"
>>>>>> +               debug="${java.debug.option}">
>>>>>> +
>>>>>> +            <bootclasspath>
>>>>>> +                <fileset dir="${hy.target}/jre/lib/boot">
>>>>>> +                    <include name="*.jar" />
>>>>>> +                </fileset>
>>>>>> +            </bootclasspath>
>>>>>> +        </javac>
>>>>>> +    </target>
>>>>>> +
>>>>>> +
>>>>>> +    <target name="run.tests">
>>>>>> +       +        <mkdir dir="${hy.tests.reports}" />
>>>>>> +
>>>>>> +        <junit fork="yes"
>>>>>> +            forkmode="once"
>>>>>> +            printsummary="withOutAndErr"
>>>>>> +            errorproperty="test.error"
>>>>>> +            showoutput="on"
>>>>>> +            dir="${hy.luni.bin.test}"
>>>>>> +            jvm="${hy.target}/jre/bin/java">
>>>>>> +
>>>>>> +            <jvmarg value="-showversion"/>
>>>>>> +
>>>>>> +            <env key="JAVA_HOME" value=""/>
>>>>>> +
>>>>>> +            <classpath>
>>>>>> +                <pathelement path="${hy.luni.bin.test}"/>
>>>>>> +            </classpath>
>>>>>> +
>>>>>> +        <test name="org.apache.harmony.tests.luni.AllTests"
>>>>>> +                haltonfailure="no"
>>>>>> +                todir="${hy.tests.reports}">
>>>>>> +                <formatter type="xml" />
>>>>>> +        </test>
>>>>>> +    </junit>
>>>>>> +    </target>
>>>>>> +   +   +    <target name="copy.resources">
>>>>>> +        <!-- Nothing for LUNI -->
>>>>>> +    </target>
>>>>>> +</project>
>>>>>> +
>>>>>>
>>>>>>
>>>>>>
>>
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.

Mime
View raw message