harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: svn commit: r883478 - in /harmony/enhanced/classlib/trunk: modules/archive/make/exclude.common modules/archive/src/test/java/org/apache/harmony/archive/tests/java/util/jar/JarFileTest.java support/src/test/java/tests/resources/hyts_flushed.jar
Date Mon, 23 Nov 2009 23:08:25 GMT
On 23/11/2009, Jesse Wilson <jessewilson@google.com> wrote:
> On Mon, Nov 23, 2009 at 2:29 PM, Tim Ellison <t.p.ellison@gmail.com> wrote:
>  > On 23/Nov/2009 20:27, Jesse Wilson wrote:
>  > > My bad for committing during the code freeze.
>  >
>  > So you're going to rollback, esp. so we don't exclude all the other
>  > tests in that type?
>  >
> I can certainly rollback the changes to the exclude.common file.
>  > > Does it make sense to limit test changes during the code freeze? I don't
>  > see
>  > > any benefit.
>  >
>  > We ensure that the tests pass and we ship the tests as part of our
>  > Milestone deliveries.  The benefit of including them in the code freeze
>  > is that we are not trying to hit a moving target and/or introducing new
>  > bugs (i.e. the same reasons the implementation code stays frozen during
>  > final test/release).
>  >
> It's dysfunctional to not checkin a test just because we don't currently
>  pass it. If anything, we need a better mechanism to manage which failures we
>  have. My personal preference is to just let them fail, and to watch Hudson
>  for unexpected failures. Suppressing a test as it gets added to avoid seeing
>  red in Hudson is bogus: our implementation is not perfect and our tools
>  should relay that back to us.

On the other hand, if there are lots of failing tests, it can be
difficult to spot tests that fail occasionally.

Test cases that are known to fail could document this, e.g. by
printing a message to say that the failure is expected.

If Harmony uses JUnit4, then the @Ignored annotation could be used for
tests that are not expected to pass; AIUI the test summary shows how
many tests have been ignored.
There are probably similar features in other unit test implementations.

View raw message