xml-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shane_Curc...@lotus.com
Subject Re: Test Infrastructure Project Proposal - followup:automatic testing daily
Date Mon, 12 Feb 2001 20:52:51 GMT
On the technical side, Sam has a very important point.  Having some sort of
self-validating automated tests that you can run as part of a magic
tinderbox/gump/whatever-we're-calling-it-now build system is vital -
especially when you link in the CVS logs.

---- you: "Sam Ruby" <rubys@nc.rr.com> wrote ----
> If you take a look at the build for xml-fop, you will see it failed.
> Because this is run regularly, I know what day the failure was
introduced.
> By replacing components with prior versions until I got a success, I was
> able to isolate the problem to a subproject.  By looking at the changes
for
> that day for that project, I was able to produce a patch which solves the
> problem - despite not being familiar with that code base.
----
While I'm not sure I fully like the Mozilla tinderbox idea of locking down
a code area when a fail happens there, it would be tremendously valuable to
have a nightly log that cross-checked any particular product's failed tests
with the CVS logs there.  Generate a report with the test fail output, plus
the CVS files that changed in that project since the last time the tests
were run and send it to the project-dev list.  That way you know one of
those specific CVS checkins was what caused the test failure. (This also
requires assuming the previous day's test passed, and/or cross-checking
which tests started failing when).

Note this doesn't necessarily address Sam's question, which is the
organizational one: presuming we had the above system (or some other cool
problem detection widget), how do we get the committers (and developers and
users) of each project to reliably step up and fix these problems in a
timely manner?  Especially considering the numerous interdependencies
between apache projects.  (I can never decide if I'd rather be at the
bottom of the food chain: like Ant or Xerces, or at the top: like FOP or
Cocoon.)

----
One other meta-point is: what are our goals?  Personally, I'd like to start
by focusing just on Apache projects in the Jakarta and XML space, to ensure
we can make quicker and more targeted progress.  Just like editors or
source code control systems, choosing how to integrate testing into your
project is a very specific (and sometimes personal) question, and I think
it'll be a lot easier to get a smaller group to buy into something than a
larger one.

- Shane




Mime
View raw message