db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ole Solberg <Ole.Solb...@Sun.COM>
Subject Re: Regression Test Failure! - Derby 383955 - Sun DBTG
Date Thu, 09 Mar 2006 10:06:46 GMT
Mike Matrigali wrote:
> I continue to really appreciate the information in these reports!
> I don't know if the following is easy or hard, but thought I would
> ask.  I just spent some time trying to track down from which change
> in the tests did a reoccuring regression come from.  This would seem
> the last piece of automated information that you could provide in
> your current automatic "attempted failure analysis".  I just spent
> time doing this by hand which involved going to the full history page
> and then doing binary search clicking down to into the details a
> couple of levels and seeing when the bug started and if it was
> consistent on that particular platform.
> It would help a lot if the current report said something like:
> test failed on xxx platform N out M times, first encountered in
> test report XXX with svn changes YYY.

I have had the same wish, and already have the basic mechanisms in 
place. I will look into integrating it with the current reporting.

> For instance I was trying to do some analysis on the stress memory
> heap issue, and found:
> test failed on linux platform 100% time (actually I didn't check
> every run, just did a few probes), first encountered in
> http://www.multinet.no/~solberg/public/Apache/Derby/Limited/testSummary-370210.html 
> 3 changes happened in that test run:
> http://www.multinet.no/~solberg/public/Apache/Derby/UpdateInfo/370210.txt
> Of which I would look at 370059 as it makes an ij change with respect to
> muti-threaded ij.
> If the reports automatically recorded this kind of info it probably 
> would be less likely for regression issues to sit around.

Ole Solberg, Database Technology Group,
Sun Microsystems, Trondheim, Norway

View raw message