couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <>
Subject Re: Script for displaying test errors and timing information
Date Mon, 16 Dec 2019 09:32:32 GMT

> On 12. Dec 2019, at 21:51, Paul Davis <> wrote:
> Hey all,
> I was poking around at Jenkins the other day trying to get a good idea
> of how much time we're spending in various parts of the build. It
> occurred to me that one good way to at least investigate our eunit
> test suite is to parse all of the generated surefire reports. I spent
> an hour or so yesterday throwing a python script together to see what
> that might look like.
> I've posted an example output and the script itself here:
> So far it shows some aggregate timing information along with the top
> few results for total suite time, total setup/teardown time, and
> longest individual test times. Not included in the output, but any
> test failures or skipped tests are also included in the output which
> has the nice benefit of showing exactly which test or tests failed
> during a run.
> So far a few interesting things jump out. First, we spend more time in
> setup/teardown fixtures than we do running actual tests which I found
> to be rather surprising. Not so surprising is that our property tests
> are taking the longest for each run.
> I thought it might be interesting to include a version of this script
> (with perhaps a somewhat more succinct output) at the end of every
> eunit run (or perhaps failed eunit run) to list out the errors for
> easier debugging of failed CI runs. And then beyond that, having it in
> the source tree would allow devs to poke around and trying to speed up
> some of the slower tests.
> Another thought was to take a look at moving the property based tests
> to a nightly build and then actually increasing their run times to
> cover more of the state space for each of those tests. That way we'd
> be doing a bit more of the normal property based testing where its
> more about long soaks to find edge cases rather than acceptance
> testing for a particular change.

I like this idea.


> So far I've just thrown this together. If there's enough of a
> consensus that we'd like to see something of this nature I can work a
> bit more on improving the script to have something that's useful both
> locally for reducing the test suite times and also for spotting errors
> that failed a CI run.
> Paul

Professional Support for Apache CouchDB:

View raw message