db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Satheesh Bandaram <sathe...@Sourcery.Org>
Subject Re: Cleaning up test failures
Date Thu, 19 Jan 2006 19:42:21 GMT
Good writeup. How do we address issue of platform specific failures? I
don't have access to SunOS or CYGWIN platforms, so I may not be able to
fix problems on those platforms.

Satheesh

David W. Van Couvering wrote:

> Hi, everyone.  Regarding keeping derbyall clean, I think there are a
> two parts to this.
>
> AWARENESS
>
> It is important for us to be aware of the current state of affairs.
> Having a web site with test results is necessary for this but, in my
> opinion, not sufficient.
>
> In order to increase awareness, I am proposing that an email is sent
> to derby-dev after each tinderbox and nightly test run sending out the
> test results.  The subject should have a standard format so that it is
> easy to filter for those of us who don't want to see these emails. 
> The subject should be different for a test run that had a failure vs.
> a clean test run so one can choose to filter out only successful test
> runs.  IMHO the subject for a failure should include attention-drawing
> text like "TEST FAILURES".
>
> Ole, would you be willing to set this up?  I am hoping it's not too
> much work.
>
> ACTION
>
> Any contributor who sees a test failure that appears to be related to
> their contribution should take action to try to resolve it.  That
> said, ultimately it is the committers who are responsible for
> maintaining the "purity" of the codeline. If a committer sees that
> there are test failures, they need to take appropriate action.  This
> can include but is not limited to:
>
> - Determining if it was one of their checkins that caused the failure,
> and working to fix the failure.
>
> - Determining whose checkins are causing the failures and directly
> contacting those individuals
>
> - As Dan suggested, placing a veto on all checkins that are not fixing
> tests until derbyall reaches an acceptable level of passes (for that
> committer). IMHO 100% is required, but we should be open to special
> cases where the bug is small to inconsequential and the level of
> effort to fix the test would be inordinately large.  Personally if a
> test consistently fails and is not fixable, it should probably be
> removed from derbyall or otherwise modified so that derbyall stays clean.
>
> Thanks,
>
> David



Mime
View raw message