db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Embretsen <John.Embret...@Sun.COM>
Subject Re: Cleaning up test failures
Date Fri, 20 Jan 2006 21:48:52 GMT
Friday, January 20, 2006, 7:19:18 PM, David W. Van Couvering wrote:

> Mike Matrigali wrote:
>> there are 2 issues, email and JIRA tracking.
>> 
>> I don't have much to say to the email discussion.  I find that if
>> some email gets sent every day I am going to ignore it unless I run
>> my tests and have a problem.  In that case I can either check the
>> email or check a web site, so either solution is fine.  The most
>> useful daily email to me would be a 1 line subject that says N nightly
>> failures across all platforms, and maybe a pointer in the email to
>> find more info.
>> 

> Are you amenable to a derby-test-results list and emails to derby-dev 
> when there is a failure?  The idea would be that these failure emails 
> would be RARE, otherwise we have a problem :)

Houston, we have a problem ;)
I don't have any hard numbers, but I was quickly browsing through
http://www.multinet.no/~solberg/public/Apache/Derby/Limited/index_all.html
looking at some random dates between April 29 2005 and January 13 2006.
I could not find a single test report where derbyall had no red "flag"
(read flag meaning that at least one test has failed) on some platform.

So, to avoid getting the E-mails you are proposing to derby-dev every
single day (which means people will ignore them after a while), we
_either_ have to become a lot better at removing test failures (I guess
by fixing the bugs or disabling the tests) than we are now, _or_ such
E-mails should only be sent e.g. when derbyall is flagged red on *all*
(or some suitable number of) platforms shown in the reports.

I guess the "failure on all platforms" option is what Mike indicated a
preference for above.


-- 
John


Mime
View raw message