db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ole Solberg <Ole.Solb...@Sun.COM>
Subject Re: Cleaning up test failures
Date Fri, 20 Jan 2006 11:15:39 GMT
David W. Van Couvering wrote:
> Hi, everyone.  Regarding keeping derbyall clean, I think there are a two 
> parts to this.
> It is important for us to be aware of the current state of affairs. 
> Having a web site with test results is necessary for this but, in my 
> opinion, not sufficient.
> In order to increase awareness, I am proposing that an email is sent to 
> derby-dev after each tinderbox and nightly test run sending out the test 
> results.  The subject should have a standard format so that it is easy 
> to filter for those of us who don't want to see these emails.  The 
> subject should be different for a test run that had a failure vs. a 
> clean test run so one can choose to filter out only successful test 
> runs.  IMHO the subject for a failure should include attention-drawing 
> text like "TEST FAILURES".
> Ole, would you be willing to set this up?  I am hoping it's not too much 
> work.

I already send such mails to myself.
As soon as there is a decision on *if* this is what the community wants 
I can start sending these to derby-dev or whatever mailing list(s) 
should be used.

> Any contributor who sees a test failure that appears to be related to 
> their contribution should take action to try to resolve it.  That said, 
> ultimately it is the committers who are responsible for maintaining the 
> "purity" of the codeline. If a committer sees that there are test 
> failures, they need to take appropriate action.  This can include but is 
> not limited to:
> - Determining if it was one of their checkins that caused the failure, 
> and working to fix the failure.
> - Determining whose checkins are causing the failures and directly 
> contacting those individuals
> - As Dan suggested, placing a veto on all checkins that are not fixing 
> tests until derbyall reaches an acceptable level of passes (for that 
> committer). IMHO 100% is required, but we should be open to special 
> cases where the bug is small to inconsequential and the level of effort 
> to fix the test would be inordinately large.  Personally if a test 
> consistently fails and is not fixable, it should probably be removed 
> from derbyall or otherwise modified so that derbyall stays clean.
> Thanks,
> David

Ole Solberg, Database Technology Group,
Sun Microsystems, Trondheim, Norway

View raw message