harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Beyer <ndbe...@apache.org>
Subject Re: svn commit: r883478 - in /harmony/enhanced/classlib/trunk: modules/archive/make/exclude.common modules/archive/src/test/java/org/apache/harmony/archive/tests/java/util/jar/JarFileTest.java support/src/test/java/tests/resources/hyts_flushed.jar
Date Tue, 24 Nov 2009 02:01:06 GMT
On Mon, Nov 23, 2009 at 5:08 PM, sebb <sebbaz@gmail.com> wrote:
> On 23/11/2009, Jesse Wilson <jessewilson@google.com> wrote:
>> On Mon, Nov 23, 2009 at 2:29 PM, Tim Ellison <t.p.ellison@gmail.com> wrote:
>>  > On 23/Nov/2009 20:27, Jesse Wilson wrote:
>>  > > My bad for committing during the code freeze.
>>  >
>>  > So you're going to rollback, esp. so we don't exclude all the other
>>  > tests in that type?
>>  >
>> I can certainly rollback the changes to the exclude.common file.
>>  > > Does it make sense to limit test changes during the code freeze? I don't
>>  > see
>>  > > any benefit.
>>  >
>>  > We ensure that the tests pass and we ship the tests as part of our
>>  > Milestone deliveries.  The benefit of including them in the code freeze
>>  > is that we are not trying to hit a moving target and/or introducing new
>>  > bugs (i.e. the same reasons the implementation code stays frozen during
>>  > final test/release).
>>  >
>> It's dysfunctional to not checkin a test just because we don't currently
>>  pass it. If anything, we need a better mechanism to manage which failures we
>>  have. My personal preference is to just let them fail, and to watch Hudson
>>  for unexpected failures. Suppressing a test as it gets added to avoid seeing
>>  red in Hudson is bogus: our implementation is not perfect and our tools
>>  should relay that back to us.
> On the other hand, if there are lots of failing tests, it can be
> difficult to spot tests that fail occasionally.
> Test cases that are known to fail could document this, e.g. by
> printing a message to say that the failure is expected.
> If Harmony uses JUnit4, then the @Ignored annotation could be used for
> tests that are not expected to pass; AIUI the test summary shows how
> many tests have been ignored.
> There are probably similar features in other unit test implementations.

We've had JUnit4 capability for a long time - we even have the
Hamcrest Matchers available. This has been the case for long enough
that I'd say we can begin depending upon it at any time. I'm not sure
what the reports that Ant generates look like, but I presume we can
modify that as we see fit.

I'd love to see the exclude files dropped in favor of JUnit @Ignore +
reporting for those ignored tests. That would be extremely helpful in
pointing out all of the excluded tests.

View raw message