harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Beyer <ndbe...@apache.org>
Subject Re: [proposal] For future milestones, expect zero test failures
Date Wed, 26 May 2010 02:00:48 GMT
On Tue, May 25, 2010 at 3:16 AM, Tim Ellison <t.p.ellison@gmail.com> wrote:
> On 25/May/2010 01:17, Nathan Beyer wrote:
>> For future milestone votes, I'd like to propose that we set the
>> expectation of zero test failures. Every time we do a milestone and I
>> run the build and tests I find myself looking at the test failures
>> (there are ALWAYS failures) and ask 'are these expected'? If the
>> failures are expected, they should be excluded and the tests should
>> just pass.
>> If this proposal is workable, immediately after this upcoming release,
>> all 'expected' failures will be pulled into the exclusion lists.
> or, preferably the tests are fixed so they pass reliably.
> I see failures in AWT/Swing that are likely caused by me not leaving the
> machine alone while I'm running the tests.  I've seen other people post
> results that I see when I've left my firewall enabled.

Yeah, I see those as well, but that's not what I'm referring to. There
are a handful of AWT/Swing tests that have been failing on Windows for
as long as I can remember. Not excluding them doesn't seem to be
getting them fixed any faster than those that are excluded.

> So, yes, I agree that getting all the tests to pass is the goal; but not
> simply by putting them into the exclude list without some further
> investigation.
> Regards,
> Tim

View raw message