db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ole Solberg <Ole.Solb...@Sun.COM>
Subject Simple "failure analysis" for regression tests. Was Re: Regression Test Failure!
Date Mon, 13 Feb 2006 10:15:23 GMT
Ole.Solberg@Sun.COM wrote:
> [Auto-generated mail]
> 
> *Derby* 377211/2006-02-12 19:45:57 CET
> *derbyall*
> 
> Failed  Tests    OK  Skip  Duration       Platform
> -------------------------------------------------------
> *Jvm: 1.5*
>     4    636    632     0   100.96%     CYGWIN_NT-5.1_i686-unknown
>    NA     NA     NA    NA               CYGWIN_NT-5.2_i686-unknown
>     1    636    635     0   102.48%     Linux-2.4.21-27.ELsmp_i686-athlon
>     2    636    634     0   102.79%     Linux-2.6.14-1.1644_FC4_i686-i686
>     2    636    634     0   93.78%     SunOS-5.10_i86pc-i386
>     2    636    634     0   114.68%     SunOS-5.10_sun4u-sparc
>     3    636    633     0   103.98%     SunOS-5.9_sun4u-sparc
>   Details in      http://www.multinet.no/~solberg/public/Apache/Derby/Limited/testSummary-377211.html


*NEW:*
>   Attempted failure analysis in
>                   http://www.multinet.no/~solberg/public/Apache/Failures/Derby/377211.html


FYI: this is trying to map the "signature" of the failure to a JIRA issue.
I keep a file with failures seen on the format

JIRA   suite/test
e.g.
800    lang/ConcurrentImplicitCreateSchema.java

and match against this.
(Should probably add "Not yet reviewed by a human!" ;-)  )

If there is no match I display something like:
"Create JIRA issue for failure in 'lang/autoincrement.sql'?"

> *Jvm: 1.4*
>     3    634    631     1   97.55%     CYGWIN_NT-5.1_i686-unknown
>     0    634    634     1   101.83%     Linux-2.4.21-27.ELsmp_i686-athlon
>    NA     NA     NA    NA               Linux-2.6.14-1.1644_FC4_i686-i686
>     1    634    633     1   97.6%     SunOS-5.10_i86pc-i386
>   Details in      http://www.multinet.no/~solberg/public/Apache/DerbyJvm1.4/Limited/testSummary-377211.html

>   Attempted failure analysis in
>                   http://www.multinet.no/~solberg/public/Apache/Failures/DerbyJvm1.4/377211.html

> 
> Changes in      http://www.multinet.no/~solberg/public/Apache/Derby/UpdateInfo/377211.txt

> 
> (All results in http://www.multinet.no/~solberg/public/Apache/index.html) 
> 


-- 
Ole Solberg, Database Technology Group,
Sun Microsystems, Trondheim, Norway

Mime
View raw message