db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Knut Anders Hatlen <knut.hat...@oracle.com>
Subject Re: concern about nightly test failures on 10.10 branch prior to release.
Date Sun, 07 Apr 2013 11:06:38 GMT
Mike Matrigali <mikem_app@sbcglobal.net> writes:

> I have been looking at nightly results of the last few weeks, with an
> eye to making sure 10.10 release does not have regressions over previous
> releases.
>
> I think in the past we have tried to get clean nightly test runs
> before making a release.  It is a problem as there are known
> intermittent errors, which make it hard to know if errors are
> regressions or not.
>
> Looking at the public nightly test runs for 10.10 I see:
>
> Java DB Testing:
> http://download.java.net/javadesktop/derby/10.10.html
> currently has 1 in a row clean recent runs, 3 out of most recent 7
> have failures.  I think it is only running on checkins, so testing
> of intermittent bugs is sparse.

I went through the failures in the above link. I found these:

1) some occurrences of DERBY-973, DERBY-3519 and DERBY-5172

2) sometimes Java 5 dumps core on Solaris SPARC

3) sometimes the compatibility tests fail to start on Solaris x64
because there is not enough memory to start a new process

4) sometimes the replication tests fail to start on Solaris x64

5) once, the JMX tests timed out when connecting to the MBean server

6) once, some network tests in derbyall failed on Java 5, 32-bit Linux,
with permission problems and UnknownHostExceptions

The bugs that cause (1) are known intermittent bugs, not new in 10.10.

(2) must be a JVM bug.

(3) is an environment problem.

I suspect (3) and (4) might be related, only that the replication tests
hide the real cause of the failure. In any case, (4) also happens in the
testing of the 10.8 and 10.9 branches, so it's most likely not a new
problem in 10.10.

(5) has also been seen once on the 10.9 branch, so it's likely not a new
problem.

That leaves us with (6), which hasn't been seen on older branches, as
far as I can see. It only happened once on the 10.10 branch. Here's a
link to the failure:

http://download.java.net/javadesktop/derby/javadb-5574721-report/javadb-5574721-3624064-details.html

The failure has the feel of a network glitch where the local host name
cannot be resolved. I wouldn't make this particular issue a blocker for
10.10 unless we see it happening more than once.

> IBM Testing:
> http://people.apache.org/~myrnavl/derby_test_results/v10_10/windows/derbyall_history.html
> http://people.apache.org/~myrnavl/derby_test_results/v10_10/windows/suites.All_history.html
> http://people.apache.org/~myrnavl/derby_test_results/v10_10/linux/derbyall_history.html
> http://people.apache.org/~myrnavl/derby_test_results/v10_10/linux/suites.All_history.html
> currently has 0 in a row clean recent runs, and I don't think there
> has been a "clean" day for 2 weeks.
>
> I have not had time to look at all the failures, to determine if they
> are regressions or not.  While not totally clean the 10.9 runs
> for the IBM Testing are much cleaner, so just using that metric it
> seems 10.10 is not ready to ship.:
> http://people.apache.org/~myrnavl/derby_test_results/v10_9/windows/suites.All_history.html


Mime
View raw message