httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dean Gaudet <dgau...@arctic.org>
Subject Re: filter API spec
Date Thu, 04 Sep 1997 03:35:40 GMT


On Tue, 2 Sep 1997, Alexei Kosut wrote:

> On Tue, 2 Sep 1997, Dean Gaudet wrote:
> 
> [...]
> 
> > There is no reason to provide bwritev() at the moment, because nothing at
> > the application level above BUFF uses it.  Nothing up there typically
> > needs to use it ... they either have a need for fully buffered output
> > (i.e. the headers, mod_include, mod_autoindex) or for completely
> > unbuffered output (i.e. default_handler, or mod_cgi).  The former
> > necessarily involve user-space memory copies because the output is being
> > generated on the fly.  The latter two involve mass output of things
> > already in memory somewhere ... and it's the latter two cases which cause
> > the writev() code in buff.c to go to work. 
> 
> Except there can be combinations of the two. Example: includes as a
> filter. default_handler does memory mapping, so creates an unbuffered
> BUFF, expecting to write out to it directly. Apache, however, has other
> plans, and sticks an includes filter on top. Now, what you really want is
> for the core default handler, which is sending out nice, big, chunks, to
> send data to the include filter in an unbuffered way, but the includes
> filter writes data in dribbles, so you want it to be buffered before
> sending it to the client.
> 
> This is pretty easy to have working, by making sure BUFF options can be
> (and are) set by all the involved parties (http_core.c, mod_include.c,
> buff.c) in the right places, and to the write (er... I mean right) BUFFs.

Yeah, but this isn't what I was referring to.  You're talking at two
different layers of the filter stack.  I'm talking about a single layer. 
In your case both layers are using BUFF in one of the ways I mentioned.

> > I want to see chunking implemented as a filter, it would vastly simplify
> > the buff.c code.  But with your current proposal we cannot do this without
> > either unnecessary memory copying or without extra system calls.
> 
> Hmm. The way Ed and I had approached it, we would have kept all the
> existing buff.c code for chunking use. But if you want chunking
> implemented as a filter at all times...

The chunking code is scary and ugly as it is.  If you left it the way it
is now then you'd be able to chunk at all layers of the filtering ... 
kind of silly imho.  I think it'll be cleaner if it's redone as a filter. 

> Well, if the chunking filter sets the BUFF below it (probably the bottom
> - the buff.c code which sends to the client) to unbuffered, and calls
> bwrite() four times for the peices of the chunk, will the current BUFF
> code use writev()? Can it be made to? Maybe we need a bwritev() after all
> (which would call writev() if it was the bottom filter, else it would
> behave the same as bwrite(), I guess).

No the current code cannot detect that.  It uses heuristics that involve
almost no state.  Here is the one heuristic that's relevant: 

#ifndef NO_WRITEV
/*
 * Detect case where we're asked to write a large buffer, and combine our
 * current buffer with it in a single writev()
 */
    if (fb->outcnt > 0 && nbyte + fb->outcnt >= fb->bufsiz) {
	return large_write (fb, buf, nbyte);
    }
#endif

i.e. given what we have in the buffer already and what we know we're about
to try to write does it make any sense for us to bother adding the new
data to our buffer?  Or should we just hand it off to writev() and start
with a fresh empty buffer?

It's just silly to try to add more heuristics when it's trivial to just
make writev() part of the filter api.  Or said another way, -1 on any
filter api that doesn't include writev.  It's even acceptable to me if you
define the API such that if the writev() vector is NULL then it simulates
writev() with multiple write()s I'll be happy ... because at least then
there is the opportunity for optimization. 

Dean



Mime
View raw message