db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Van Couvering <David.Vancouver...@Sun.COM>
Subject Re: Regression testing
Date Tue, 10 May 2005 18:00:48 GMT

Ole Solberg wrote:

> Andrew McIntyre wrote:
>> On May 4, 2005, at 2:44 PM, Ole Solberg wrote:
>>> We also build and test on a few platforms daily and could provide 
>>> those results.
>>> My level of ambition would be to just send out the results without 
>>> any deep analysis. (Just catching and filtering obvious local 
>>> setup/enviroment blunders etc.)
>>> I think communicating daily regression test results could be a good 
>>> way to present the state of Derby.
>> It's great to hear that other derby-dev'rs are building Derby nightly 
>> and running the tests! I think it would be a very good thing to be 
>> sharing test results, but I'm a bit concerned about sending them to 
>> derby-dev itself. I personally feel that nightly automated mail to the 
>> list would simply decrease the signal-to-noise ratio on the list and 
>> be a bit of a nuisance (and ultimately not likely to be read). But 
>> there are alternatives to sending nightly test results to the list, 
>> like posting them in a specific location on the Derby website, as I'm 
>> currently doing with the doc/javadoc build. Or we could have a page on 
>> the Derby website with links to locations where derby-dev'rs are 
>> publishing their test results.
> I agree that that having a page on the Derby website linking to the 
> actual test reports would be a much better solution than "polluting" 
> derby-dev with lots of e-mails.
> Anyone with neccessary rights willing to create such a page?

Again, I'm not sure why there would be "lots of emails" for a nightly 
build.  Also, if we email failures, that would be great, as I know I 
personally won't be checking the test web site on a regular basis.

>> In an ideal situation, it would be great to have a tinderbox approach, 
>> as David suggested, with constantly active build/test cycles running. 
>> But there's always the complicated question of who's going to provide 
>> the hardware and put the box out on the net for all to see when going 
>> that route.
> Which tests should be included in such an approach?
> What must the maximum turnaround time for this be to be considered useful?

Well, it seems to me you just have a cycle of "pull, build, full 
regresssions, post results."  If we have a powerful enough machine, it 
shouldn't take that long, and we'd have fresh results every, say, four 
hours.  Not bad!

If we want a faster turnaround, we could, as you suggest, identify a 
subset of tests.

>> andrew
> -- Ole

View raw message