db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David W. Van Couvering" <David.Vancouver...@Sun.COM>
Subject Re: Regression Test Harness handling of "duration" field
Date Wed, 08 Feb 2006 18:40:46 GMT
+1 on the trigger, I had suggested that earlier.

David

Bryan Pendleton wrote:
> David W. Van Couvering wrote:
> 
>> My understanding it's the wall clock time. 
> 
> 
>>>   derbyall   630 6 624 0
>>>     Duration   45.6%
> 
> 
> OK. So it's saying that this particular run of 'derbyall'
> took 46.2% of the wall clock time that 'derbyall' took
> on August 2, 2005.
> 
> That makes sense.
> 
> Given that this particular run failed badly, we probably
> don't care about the Durations, then.
> 
> But in general, we'd probably want to keep our eyes open
> for Duration values that started to get significantly
> higher than 100%, because that would mean that we might
> have accidentally introduced a performance regression.
> 
> Perhaps we could have some sort of trigger, so that if a
> suite experienced a duration of, say, 150%, that was
> treated as a regression failure, even if all the tests in
> that suite passed?
> 
> thanks,
> 
> bryan
> 

Mime
View raw message