harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geir Magnusson Jr <g...@pobox.com>
Subject Re: luni test script tweaks
Date Wed, 08 Mar 2006 00:00:32 GMT


Tim Ellison wrote:
> Geir Magnusson Jr wrote:
>>
>> Tim Ellison wrote:
>>> Geir Magnusson Jr wrote:
>>>> Tim Ellison wrote:
>>>>> Geir Magnusson Jr wrote:
>>>>>> do you want to discuss what you are undoing?
>>>>> I assume the monster diff is due to EOL differences or something...
>>>>>
>>>>> 1. I've make the test report location a variable
>>>> I thought you pointed it back into luni?
>>> Nope, see  http://svn.apache.org/viewcvs?rev=383948&view=rev
>>>
>>>>> 2. I've undone somebody else's undoing of the test suite invocation
>>>> Right, because we didn't get the same info from doing the suite as we
>>>> did form running individual tests, IIRC.
>>> Such as...?  Launching the VM for every individual test case is not
>>> helpful here.
>> forkmode="once"
> 
> Yep

Meaning that forkmode="once" solves that problem.  So this isn't an issue.

> 
>>>> Since the suite was only a set of tests (no extra setup)
>>> what?
>> IOW, you don't do anything else.  It's just a bag.
> 
> Once we get HARMONY-57 tests installed we will wrap the suite in the
> exclusions list support code, and can apply any other logic to select
> which tests are run, check for performance regressions, etc.  It will
> still look like one JUnit suite to the Ant task.  If we build the suite
> from <batchtest> including *Test.java then I don't think we can do that?

I guess we can see what's proposed and accept or shoot it down then. 
I'm trying to limit the number of touch and maintenance points, and make 
the reporting richer.

Maybe an ant task that reads meta-data (an exclusion list, a grouping, 
etc) and runs junit with suites created on the fly.... hm.  So tests can 
be just dropped in, and there is some default suite that just picks them 
up automatically...  Hmmm...

[SNIP]

> 
> Just looking at a report from a test suite, I can see the individual
> tests that were run.  Is that what you meant?
> 
> Here's what I did:
> 
> cd make
> ant   (to build the classlib code)
> copy across the VM into deploy, and XML jars into lib/boot
> ant -f build-test.xml test-luni,gen-report
> 
> then browse ..\build\test_report\html\index.html

So compare these two... First as is now in SVN :

http://people.apache.org/~geirm/test_report_alltests/html/

And as it was in SVN (I just locally reverted the build.xml in 
luni/common/ to do this...)

http://people.apache.org/~geirm/test_report_indiv/html/

The latter has more information.  It lets you view by package, and by 
test class, which is a useful unit of measure.  It also churns out more

We may be able to get the same out of using a test suite, but IIRC, I 
tried a few week ago when I set this up, and wasn't able to...  It's 
hard to imagine that there isn't a way...

geir

> 
> Regards,
> Tim
> 
>>>>>> tellison@apache.org wrote:
>>>>>>> Author: tellison
>>>>>>> Date: Tue Mar  7 10:08:47 2006
>>>>>>> New Revision: 383950
>>>>>>>
>>>>>>> URL: http://svn.apache.org/viewcvs?rev=383950&view=rev
>>>>>>> Log:
>>>>>>> Use the test suite, and put the results in the reporting dir
>>>>>>>
>>>>>>> Modified:
>>>>>>>  
>>>>>>> incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Modified:
>>>>>>> incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> URL:
>>>>>>> http://svn.apache.org/viewcvs/incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml?rev=383950&r1=383949&r2=383950&view=diff
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ==============================================================================
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---
>>>>>>> incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml
>>>>>>>
>>>>>>>
>>>>>>> (original)
>>>>>>> +++
>>>>>>> incubator/harmony/enhanced/classlib/trunk/modules/luni/make/common/build.xml
>>>>>>>
>>>>>>>
>>>>>>> Tue Mar  7 10:08:47 2006
>>>>>>> @@ -1,111 +1,99 @@
>>>>>>> -<?xml version="1.0" encoding="UTF-8"?>
>>>>>>> -<!--
>>>>>>> -    Copyright 2006 The Apache Software Foundation or its licensors,
>>>>>>> as applicable.
>>>>>>> -  -    Licensed under the Apache License, Version 2.0 (the
>>>>>>> "License");
>>>>>>> -    you may not use this file except in compliance with the
License.
>>>>>>> -    You may obtain a copy of the License at
>>>>>>> -  -       http://www.apache.org/licenses/LICENSE-2.0
>>>>>>> -  -    Unless required by applicable law or agreed to in writing,
>>>>>>> software
>>>>>>> -    distributed under the License is distributed on an "AS IS"
>>>>>>> BASIS,
>>>>>>> -    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
or
>>>>>>> implied.
>>>>>>> -    See the License for the specific language governing permissions
>>>>>>> and
>>>>>>> -    limitations under the License.
>>>>>>> --->
>>>>>>> -
>>>>>>> -<project name="Common_LUNI_Build">
>>>>>>> -   -    <target name="compile.java" description="Compile
LUNI java
>>>>>>> code">
>>>>>>> -        <echo message="Compiling LUNI classes from
>>>>>>> ${hy.luni.src.main.java}" />
>>>>>>> -       -        <mkdir dir="${hy.luni.bin.main}" />
>>>>>>> -
>>>>>>> -        <javac sourcepath=""
>>>>>>> -            srcdir="${hy.luni.src.main.java}"
>>>>>>> -            destdir="${hy.luni.bin.main}"
>>>>>>> -            source="${source.ver}"
>>>>>>> -            debug="${java.debug.option}">
>>>>>>> -
>>>>>>> -            <bootclasspath>
>>>>>>> -                <fileset dir="${hy.target}/jre/lib/boot">
>>>>>>> -                    <include name="*.jar" />
>>>>>>> -                </fileset>
>>>>>>> -            </bootclasspath>
>>>>>>> -        </javac>
>>>>>>> -    </target>
>>>>>>> -   -    <target name="build.jar">
>>>>>>> -        <jar destfile="${hy.target}/jre/lib/boot/luni.jar"
>>>>>>> manifest="${hy.luni}/META-INF/MANIFEST.MF">
>>>>>>> -            <fileset dir="${hy.luni.bin.main}" />
>>>>>>> -        </jar>
>>>>>>> -    </target>
>>>>>>> -
>>>>>>> -   -    <target name="compile.tests">
>>>>>>> -        <echo message="Compiling LUNI tests from
>>>>>>> ${hy.luni.src.test.java}" />
>>>>>>> -
>>>>>>> -        <mkdir dir="${hy.luni.bin.test}" />
>>>>>>> -
>>>>>>> -        <javac srcdir="${hy.luni.src.test.java}"
>>>>>>> -               destdir="${hy.luni.bin.test}"
>>>>>>> -               sourcepath=""
>>>>>>> -               source="${source.ver}"
>>>>>>> -               debug="${java.debug.option}">
>>>>>>> -
>>>>>>> -            <bootclasspath>
>>>>>>> -                <fileset dir="${hy.target}/jre/lib/boot">
>>>>>>> -                    <include name="*.jar" />
>>>>>>> -                </fileset>
>>>>>>> -            </bootclasspath>
>>>>>>> -        </javac>
>>>>>>> -    </target>
>>>>>>> -
>>>>>>> -
>>>>>>> -    <target name="run.tests">
>>>>>>> -       -        <property name="test.dir"
>>>>>>> value="../../../../target/test_report" />
>>>>>>> -        <mkdir dir="${test.dir}" />
>>>>>>> -
>>>>>>> -        <junit fork="yes"
>>>>>>> -            forkmode="once"
>>>>>>> -            printsummary="withOutAndErr"
>>>>>>> -            errorproperty="test.error"
>>>>>>> -            showoutput="on"
>>>>>>> -            dir="${hy.luni.bin.test}"
>>>>>>> -            jvm="${hy.target}/jre/bin/java">
>>>>>>> -
>>>>>>> -            <jvmarg value="-showversion"/>
>>>>>>> -
>>>>>>> -            <env key="JAVA_HOME" value=""/>
>>>>>>> -
>>>>>>> -            <classpath>
>>>>>>> -                <pathelement path="${hy.luni.bin.test}"/>
>>>>>>> -            </classpath>
>>>>>>> -
>>>>>>> -            <!--
>>>>>>> -            <test name="org.apache.harmony.tests.luni.AllTests"
>>>>>>> -                haltonfailure="no"
>>>>>>> -                todir="${test.dir}">
>>>>>>> -                <formatter type="xml" />
>>>>>>> -            </test>
>>>>>>> -            -->
>>>>>>> -           -            <formatter type="xml" />
>>>>>>> -            <batchtest todir="${test.dir}">
>>>>>>> -                <fileset dir="../../src/test/java">
>>>>>>> -                    <include name="**/*Test.java"/>
>>>>>>> -                </fileset>
>>>>>>> -            </batchtest>
>>>>>>> -
>>>>>>> -        </junit>
>>>>>>> -       -    </target>
>>>>>>> -   -   -    <target name="copy.resources">
>>>>>>> -        <!-- Nothing for LUNI -->
>>>>>>> -    </target>
>>>>>>> -</project>
>>>>>>> -
>>>>>>> +<?xml version="1.0" encoding="UTF-8"?>
>>>>>>> +<!--
>>>>>>> +    Copyright 2006 The Apache Software Foundation or its licensors,
>>>>>>> as applicable.
>>>>>>> +  +    Licensed under the Apache License, Version 2.0 (the
>>>>>>> "License");
>>>>>>> +    you may not use this file except in compliance with the
License.
>>>>>>> +    You may obtain a copy of the License at
>>>>>>> +  +       http://www.apache.org/licenses/LICENSE-2.0
>>>>>>> +  +    Unless required by applicable law or agreed to in writing,
>>>>>>> software
>>>>>>> +    distributed under the License is distributed on an "AS IS"
>>>>>>> BASIS,
>>>>>>> +    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
or
>>>>>>> implied.
>>>>>>> +    See the License for the specific language governing permissions
>>>>>>> and
>>>>>>> +    limitations under the License.
>>>>>>> +-->
>>>>>>> +
>>>>>>> +<project name="Common_LUNI_Build">
>>>>>>> +   +    <target name="compile.java" description="Compile
LUNI java
>>>>>>> code">
>>>>>>> +        <echo message="Compiling LUNI classes from
>>>>>>> ${hy.luni.src.main.java}" />
>>>>>>> +       +        <mkdir dir="${hy.luni.bin.main}" />
>>>>>>> +
>>>>>>> +        <javac sourcepath=""
>>>>>>> +            srcdir="${hy.luni.src.main.java}"
>>>>>>> +            destdir="${hy.luni.bin.main}"
>>>>>>> +            source="${source.ver}"
>>>>>>> +            debug="${java.debug.option}">
>>>>>>> +
>>>>>>> +            <bootclasspath>
>>>>>>> +                <fileset dir="${hy.target}/jre/lib/boot">
>>>>>>> +                    <include name="*.jar" />
>>>>>>> +                </fileset>
>>>>>>> +            </bootclasspath>
>>>>>>> +        </javac>
>>>>>>> +    </target>
>>>>>>> +   +    <target name="build.jar">
>>>>>>> +        <jar destfile="${hy.target}/jre/lib/boot/luni.jar"
>>>>>>> manifest="${hy.luni}/META-INF/MANIFEST.MF">
>>>>>>> +            <fileset dir="${hy.luni.bin.main}" />
>>>>>>> +        </jar>
>>>>>>> +    </target>
>>>>>>> +
>>>>>>> +   +    <target name="compile.tests">
>>>>>>> +        <echo message="Compiling LUNI tests from
>>>>>>> ${hy.luni.src.test.java}" />
>>>>>>> +
>>>>>>> +        <mkdir dir="${hy.luni.bin.test}" />
>>>>>>> +
>>>>>>> +        <javac srcdir="${hy.luni.src.test.java}"
>>>>>>> +               destdir="${hy.luni.bin.test}"
>>>>>>> +               sourcepath=""
>>>>>>> +               source="${source.ver}"
>>>>>>> +               debug="${java.debug.option}">
>>>>>>> +
>>>>>>> +            <bootclasspath>
>>>>>>> +                <fileset dir="${hy.target}/jre/lib/boot">
>>>>>>> +                    <include name="*.jar" />
>>>>>>> +                </fileset>
>>>>>>> +            </bootclasspath>
>>>>>>> +        </javac>
>>>>>>> +    </target>
>>>>>>> +
>>>>>>> +
>>>>>>> +    <target name="run.tests">
>>>>>>> +       +        <mkdir dir="${hy.tests.reports}" />
>>>>>>> +
>>>>>>> +        <junit fork="yes"
>>>>>>> +            forkmode="once"
>>>>>>> +            printsummary="withOutAndErr"
>>>>>>> +            errorproperty="test.error"
>>>>>>> +            showoutput="on"
>>>>>>> +            dir="${hy.luni.bin.test}"
>>>>>>> +            jvm="${hy.target}/jre/bin/java">
>>>>>>> +
>>>>>>> +            <jvmarg value="-showversion"/>
>>>>>>> +
>>>>>>> +            <env key="JAVA_HOME" value=""/>
>>>>>>> +
>>>>>>> +            <classpath>
>>>>>>> +                <pathelement path="${hy.luni.bin.test}"/>
>>>>>>> +            </classpath>
>>>>>>> +
>>>>>>> +        <test name="org.apache.harmony.tests.luni.AllTests"
>>>>>>> +                haltonfailure="no"
>>>>>>> +                todir="${hy.tests.reports}">
>>>>>>> +                <formatter type="xml" />
>>>>>>> +        </test>
>>>>>>> +    </junit>
>>>>>>> +    </target>
>>>>>>> +   +   +    <target name="copy.resources">
>>>>>>> +        <!-- Nothing for LUNI -->
>>>>>>> +    </target>
>>>>>>> +</project>
>>>>>>> +
>>>>>>>
>>>>>>>
>>>>>>>
> 

Mime
View raw message