harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Fedotov" <alexei.fedo...@gmail.com>
Subject Re: [testing] Tests scores on http://harmonytest.org Was: [DRLVM] General stability
Date Fri, 10 Nov 2006 22:28:35 GMT

I like your approach to result comparison. 10% can be default value
for some form field - anyone can change it if needed.

As for test execution time reported by JUnit, it is applicable for
stress tests as well if we gradually increase a load over time. Though
using ttime field for stress tests and documentation rankings is not
quite conventional.

Probably more conventional would be to parse
<system-out><![CDATA[]]></system-out> for some metric. Does the site
already parse TEST-* files?

Thank you, Alexei

On 11/10/06, Anton Luht <anton.luht@gmail.com> wrote:
> Hello, Alexei,
> > I have related question. How can we improve http://harmonytest.org to
> > make it possible to publish not just pass, fail, or error but numeric
> > test scores?
> Easily - test results in JUnit reports have 'time' property -
> execution time in seconds. We can import and show them in the results.
> What else is needed? Maybe add something like 'show regressions' to
> the the 'compare runs' page? For example, show tests that increased
> execution time by more than 10% sorted by increase rate desc?
> --
> Regards,
> Anton Luht,
> Intel Java & XML Engineering

Thank you,

View raw message