db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Pendleton <bpendle...@amberpoint.com>
Subject Re: Regression Test Harness handling of "duration" field
Date Wed, 08 Feb 2006 18:31:25 GMT
David W. Van Couvering wrote:
> My understanding it's the wall clock time. 

>>   derbyall   630 6 624 0
>>     Duration   45.6%

OK. So it's saying that this particular run of 'derbyall'
took 46.2% of the wall clock time that 'derbyall' took
on August 2, 2005.

That makes sense.

Given that this particular run failed badly, we probably
don't care about the Durations, then.

But in general, we'd probably want to keep our eyes open
for Duration values that started to get significantly
higher than 100%, because that would mean that we might
have accidentally introduced a performance regression.

Perhaps we could have some sort of trigger, so that if a
suite experienced a duration of, say, 150%, that was
treated as a regression failure, even if all the tests in
that suite passed?

thanks,

bryan


Mime
View raw message