harmony-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Varlamov (JIRA)" <j...@apache.org>
Subject [jira] Created: (HARMONY-4842) [bti] Unreliable status calculation for snapshot testing results
Date Mon, 24 Sep 2007 09:06:50 GMT
[bti] Unreliable status calculation for snapshot testing results
----------------------------------------------------------------

                 Key: HARMONY-4842
                 URL: https://issues.apache.org/jira/browse/HARMONY-4842
             Project: Harmony
          Issue Type: Bug
          Components: build - test - ci
            Reporter: Alexey Varlamov


The snapshot status page [1] wrongly accounts some suites PASSED. It happens when suite is
expected to generate JUnit-style report and some tests crash or cannot compile and build run
aborts after running only part of suite. In this case those few successfully run tests are
published and suite status is shown green on the page [1]. 
This in particular explains wrong data for DRLVM tests on Windows [2], which in reality is
caused by compilation errors in cunit tests.

The most reliable way would be to generate status.txt always and rely on it when calculating
status, while look for JUnit statistics only to get pass rate.

[1] http://people.apache.org/~mloenko/snapshot_testing/script/snapshots_summary.html
[2] http://people.apache.org/~mloenko/snapshot_testing/script/r577501/index.html

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message