httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Stein <>
Subject Re: mod_dav overhaul
Date Sun, 01 Jun 2003 09:45:18 GMT
On Thu, May 29, 2003 at 08:38:31AM -0500, Ben Collins-Sussman wrote:
> By the way, for those interested in reading about our "best pool
> practices" philosophy, take a look at Subversion's HACKING guidelines
> -- at the section called "APR pool usage conventions".  These are the
> magic formulas that we'd like to spread into mod_dav:

Ideally, this should apply to *all* Apache modules. Historically, these
practices were never recorded, nor emboded in general practice. Thus,
mod_dav fell down on them. Separately, OtherBill mentioned needing to apply
these to mod_autoindex. Definitely. And other modules could go with it,

> > >2. make all mod_dav_svn callbacks take pools.  But then we wrap these
> > >   callbacks with wrappers that *don't* take pools, so mod_dav_svn can
> > >   still plug-in to mod_dav.
> > 
> > Why - in the sense that httpd-2.1 forward breaks binary compatibility?
> > Wouldn't it make more sense to extend mod_dav_svn (and other dav
> > backends) going forward?  We aren't troubling ourselves with that sort
> > of compatibility for httpd-2.2, in that we will just get the API 'right' as
> > of each major/minor bump.  That means code must be 'adjusted' for
> > the next release anyways.
> We don't want to force Subversion to depend on the httpd-2.1 branch.

Exactly. Subversion is targeted towards Apache 2.0.x. The intent is to
implement the improved pool handling in the 2.1 branch, then backport that
work to the 2.0 branch.

As Justin states, we're pretty well screwed if we alter our dependency graph
to depend upon Apache 2.1.

> > >3. use the API versioning which is present in httpd-2.0 (and later) to
> > >   define two back-end interfaces for mod_dav. The current interface
> > >   is v1, and the new "pool accepting" interface becomes v2.  mod_dav
> > >   is updated so that it can use either interface.  mod_dav_svn is
> > >   updated to supply both: the old wrapper-based form, and the new
> > >   form. This will allow a mix/match between httpd-2.0.X (and 2.X.Y)
> > >   and mod_dav_svn.
> > 
> > I'm mostly asking, is this a real win?
> This is Greg Stein's department;  I'm not familiar with the political
> implications of our various options.   ("Do we make v1 and v2 provider
> interfaces available to the 2.0 branch?  Or just declare that mod_dav
> has a new API in httpd-2.1?")  I'll let him address this topic.

The eventual target for these fixes is the 2.0 branch (via initial work on
the 2.1 branch). I built mod_dav over 4.5 years ago. The concept of "proper
pool usage" simply was not a factor in that design. After working on
Subversion for the past few years and dealing with scalability within the
pool-based design paradigm, I (and the SVN project) have learned a ton. This
isn't just the SVN project trying to jam some fixes in, but it is about
fixing some (IMO) broken code that I built into mod_dav. We've got some of
the same elsewhere in Apache, but mod_dav has a few areas which are very
iterative, and that's actually kind of unusual within Apache, which is why
we don't normally see it. About three years ago, we rebuilt some
mod_autoindex stuff with subrequests because of memory consumption -- that
was basically an aspect of this pool usage stuff that we're talking about.

In any case, back to the original question: the set-up and use of multiple
versions of the mod_dav back-end API are to enable full mix/match within the
2.0 code space. Modules built to the 2.0 mod_dav API ("v1") need to continue
to work. And we can keep them working. But we also want to improve operation
for the backends that can be smarter about it. Thus, we want to introduce
the "v2" interface. The biggest question for v2 is whether we do a pragmatic
"fix >this< and >that<" and be done with it, or whether we try and manage a
larger revamp. That will mostly be up to the amount of time the volunteers
have. But this action will certainly demonstrate that we can have multiple
APIs, which means that we could defer some of the improvements to a future,
v3 interface.

> But at the moment, it's outside the scope o our immediate goals to
> drag every mod_dav_* backend kicking-and-screaming into a new v2
> interface.  We just want to make mod_dav use pools efficiently in
> httpd-2.0, and that's a relatively short-term project.

Right. And it is important to note that we can't really drag "every" module
because we don't have a full enumeration of them. We know the three open
source modules, but I know that IBM, Oracle, and Apple all use mod_dav in
their products (do they have custom backends?). At the OSCOM conference
here, I spoke with a guy that had a custom module. Personally, I consider
that mod_dav API to be as sancrosanct as the rest of the Apache 2.0 API.
This is also why I/Justin were so gung-ho on introducing the versioned
plugin provider interface (thanks, Justin!) to the mod_dav and auth systems.

All that said: it currently appears that the primary work will be on
transforming the PROPFIND response from a buffered response to a streamy
response. That can pretty well be done without any API change. The original
buffering was done so that we could return 500 errors or somesuch, but in
the new model, we'll simply bury those down in the multistatus response. I
have no idea whether clients will be able to properly detect that, but they
"should" and that pattern of response should be amenable.


Greg Stein,

View raw message