httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Collins-Sussman <>
Subject mod_dav overhaul
Date Wed, 28 May 2003 18:12:08 GMT

Hi, I'm a new subscriber to this list, but not a complete stranger
either.  My name is Ben Collins-Sussman, and I'm one of the original
Subversion developers.  I'm also an APR committer, and well... I know
a whole lot of people on this list, and met many more of you at the
last Apachecon.

I'm writing because I plan to do some major overhauling of mod_dav.
No need to fear, because I'm essentially a disciple of gstein.  :-)
I'm reasonably familiar with mod_dav code already, because I'm
intimate with the mod_dav_svn provider (Subversion's main server.)

The problem here is that mod_dav is very inefficient about its pool
usage.  Back in the subversion project, we've developed a set of "best
practices" regarding pool usage;  it took us a couple years to
discover and refine these rules.  But mod_dav isn't following these
practices, as it was written long ago.  As a result, mod_dav is having
serious problems scaling to large Subversion repositories.

For example: if a repository directory contains thousands (or even
hundreds) of entries, a PROPFIND of depth 1 will cause the httpd
process to grow absolutely enormous, sometimes using up all available

Greg Stein has proposed a 4-part revamp plan for mod_dav in httpd-2.1,
which I'd like to run by folks:

1. revamp mod_dav's use of pools internally.

   This means adding pool arguments to the various "layers" of
   function calls within mod_dav.  Each caller controls subpool
   creation, and possibly passes a subpool into a subordinate function
   call.  Whereever there are loops that allocate temporary memory, we
   use (and *clear*) a subpool within the loop.  These changes are
   made up to the point of the provider backend;  but we halt there,
   and do not pass pools into provider callbacks.

2. make all mod_dav_svn callbacks take pools.  But then we wrap these
   callbacks with wrappers that *don't* take pools, so mod_dav_svn can
   still plug-in to mod_dav.

3. use the API versioning which is present in httpd-2.0 (and later) to
   define two back-end interfaces for mod_dav. The current interface
   is v1, and the new "pool accepting" interface becomes v2.  mod_dav
   is updated so that it can use either interface.  mod_dav_svn is
   updated to supply both: the old wrapper-based form, and the new
   form. This will allow a mix/match between httpd-2.0.X (and 2.X.Y)
   and mod_dav_svn.

4. we can eventually backport these changes from httpd-2.1 to the
   httpd-2.0 branch;  it still maintains compatibility with v1
   providers, such as mod_dav_fs.


View raw message