httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Champion <champio...@gmail.com>
Subject Re: Automated tests
Date Tue, 03 Jan 2017 20:05:58 GMT
On 12/30/2016 02:55 PM, Stefan Fritsch wrote:
> Yes, httpd lacks unit tests. One problem is that many APIs depend on very
> complex structs like request_rec, conn_rec, server_conf, etc. In order to
> write unit tests for such APIs, one would need to write quite a bit of
> infrastructure to set these things up. I think it would be worth the effort,
> but it's not a small task. As there does not seem to be anybody with enough
> spare time to do it, one could possibly ask someone (CII?) for funding.
>
> A possible approach would be to compile the unit tests in the server and
> execute them on startup if a special define is given (like the various DUMP_*
> defines). Not sure how to get access to all the static helper function for unit
> tests, though. Unless one would somehow include the tests in the same .c file.

That's an interesting idea. To riff on that a little bit: I've seen some 
questions on #httpd recently about the shared-library build for 
libhttpd, which IIUC only exists on Windows at the moment. It seems like 
having a libhttpd would simplify building a unit test executable... can 
anyone point me to the history behind the removal of that feature?

> If the test suite would be easier to run, maybe more people would submit
> tests. Is there a reason why the test suite is in a separate repository? Would
> it help if it was moved into the normal httpd repo? Would it make sense to
> include it in the release tarballs, possibly including the necessary non-
> standard perl modules? And include it in the makefiles in a way that a user can
> install a set of standard perl modules (from a distribution or cpan) and then
> call "make test" to start it. What is in the test/ dir in the httpd repo right
> now seems mostly useless and could probably be removed.

My personal end goals are
- to be able to perform the standard `make && make check` invocation 
without installation (this was discussed with a user in another dev@ thread)
- to have a bugfix/feature *and* its related tests in the same commit or 
backported patchset

So, to that end, I'd like to see the test suite eventually move into the 
httpd repo. I think I can start on my first goal without that, though 
(and I plan to start looking at that soon). That will hopefully give us 
time to discuss any possible fallout of merging the two codebases, while 
giving us some of the benefits in the meantime.

> Another idea to make writing tests more attractive could be to somehow include
> it in the backporting policy. For example, if there is a test for a new
> feature (positive and error handling) or a bug fix, we could require only two
> +1s for a backport.

I like this idea too.

> Another thing that is missing: A buildbot that builds current trunk (and
> possibly 2.x branches) and runs the test suite and alerts the dev list of
> regressions. I guess this "just" needs a volunteer to set it up and document
> it and the ASF would provide the infrastructure.

+1. This is a prerequisite to having a nice release cadence, IMHO.

--Jacob

Mime
View raw message