Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: harmony-dev@incubator.apache.org
Received-SPF: pass (herse.apache.org: domain of t.p.ellison@gmail.com
 designates 66.249.92.170 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:x-enigmail-version:content-type:content-transfer-encoding;
        b=XbpwDxkfqwHvhbvdXvXOM9TiGDbWbGIO0xa3i/kLR6PmCidhand80SS4sT20OncHnnV07oraDAOf3hq9jvfAKP02yHzVa5OQLv+9Cq6lE41q9srLNRvHm32MzkH6A1J+Nu8l5vZBqbIC5h65enzT3iPK8+SYsGCI5LwiQVAk5Aw=
Message-ID: <455B8D87.3000603@gmail.com>
Date: Wed, 15 Nov 2006 21:58:31 +0000
From: Tim Ellison <t.p.ellison@gmail.com>
User-Agent: Thunderbird 1.5.0.8 (Windows/20061025)
MIME-Version: 1.0
To: harmony-dev@incubator.apache.org
Subject: [classlib][testing] Showing test failures (Re: [drlvm] New
 regression:
 java.lang.ClassGenericsTest4)
References: <200611150401.28037.gshimansky@gmail.com>
	 <f944d83d0611150038x246390f1ob32289ee4c1fff42@mail.gmail.com>
	 <ejf2o0$2bc$1@sea.gmane.org>
	 <f944d83d0611150521i4f75ba4as732428ea9c1cec8a@mail.gmail.com>
	 <f944d83d0611150531y6fe3e608vce8a4f0b8e36e860@mail.gmail.com>
	 <ejf551$asj$1@sea.gmane.org>
	 <f944d83d0611150857m56dc3666j6bdb785c484001d0@mail.gmail.com>
	 <f944d83d0611150925j5c48f9a2ia3d7603a089513e5@mail.gmail.com>
	 <783bf8b0611151000x6c85a50ag30c72e99a4e0400d@mail.gmail.com>
	 <c30d70960611151026r32a2de75gc221ea5c57d51db3@mail.gmail.com>
 <51d555c70611151037i34e641b9xcc4539403c80dbc8@mail.gmail.com>
In-Reply-To: <51d555c70611151037i34e641b9xcc4539403c80dbc8@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Rana Dasgupta wrote:
> I think that a problem with the junit tests is that some failures spit out
> to the console, but show up in the test run results as passed. I find this
> very confusing. So unless you are watching all the time, you can miss them.

Hmm, this doesn't sound right.  I've not seen classlib tests fail but be
reported in the html report as passed.  Is that what you meant? (I
realize your mail subject was [drlvm] but just checking.)

One confusing aspect is that the classlib ant build fails if you run the
tests 'globally', but passes if you run the tests in a single module.

We could fix that by making the current test sequence an internal target
(for the 'global' test), and checking for failures after a 'modular' test.

e.g.

Index: build.xml
===================================================================
--- build.xml	(revision 474801)
+++ build.xml	(working copy)
@@ -41,8 +41,19 @@

     <target name="build" depends="compile-java, copy-resources,
build-jar" />

-    <target name="test" depends="build, compile-tests, run-tests" />
+    <target name="test" depends="-test">
+        <fail message="Some tests failed">
+            <condition>
+                <or>
+                    <isset property="test.failures" />
+                    <isset property="test.errors" />
+                </or>
+            </condition>
+        </fail>
+    </target>

+    <target name="-test" depends="build, compile-tests, run-tests" />
+
     <target name="clean">
         <delete file="${hy.jdk}/jre/lib/boot/beans.jar" />
         <delete file="${hy.jdk}/jre/lib/boot/beans-src.jar" />


That would be more intuitive to me, so unless anyone objects I'll do it.

Regards,
Tim

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.