httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject RE: cvs commit: apache-2.0/src/main http_core.c http_protocol.c
Date Tue, 03 Oct 2000 00:19:01 GMT
On Mon, 2 Oct 2000, William A. Rowe, Jr. wrote:

> > From: []
> > Sent: Monday, October 02, 2000 6:46 PM
> > 
> > I disagree completely.  If mod_random is written to re-arrange buckets,
> > then it is broken.  It is completely impossible to determine how a handler
> > will arrange the content in a brigade.  
> Still disagree here.  Another fine example.  Let us consider performance from
> this perspective... we have a filter that takes an inordinate amount of cpu
> per iteration, and no significant time per byte.  Bill's filter, in this case,
> would be a great benefit to drop the number of iterations through such a filter.
> Excuse me for not being able to invent a specific example.

If you can't come up with a real example, that's hand-waving.  I would
really like to see an actual example before adding in a special buffering

> > Filters can not be written to expect data in any particular
> > format.  Filters have to accept buckets of data, and they have to work
> > with the data, not the buckets.  Otherwise the result of those filters
> > will be unpredictable.
> No... filters cannot expect data to be in a particular format, but need
> to be able to identify the data and warp it appropriately.  Perhaps my
> long term vision of filters is broader than some folks, but I've always
> pictured a filter picking up on headers_out and deciding that they are
> something else, and the filter is the means to make that something else.

If I understand what you are saying, you want filters to be able to
determine where data came from.  The only way to expect this, is for the
filter and the module handler to be linked.  In that case, I would expect
a new bucket type (we really need to abstract those types out), which most
filters treat generically using the read function.  However, the filter
that knows about that bucket type would be able to treat those buckets

> > There are only two reasons IMHO to buffer above the core filter:
> > 
> > 1)  The ap_r* functions should do some buffering, so that we make as few
> > trips down the stack as possible.  This is an optimization and should wait
> > to be done.  I expect there will need to be some logic to ensure that we
> > don't try to buffer too much or at inappropriate times.
> ap_r* is going away, correct?  It's legacy, and I'm not too concerned.

ap_r* is being frowned upon, but not really going away.  It is not the
optimal way to do things, but I suspect it will be in the server for a
long time yet.

> > 2)  The filter requires it.  This is something like our favorite
> > mod_content_length example, where the filter requires that it has all of
> > the data before it passes down the stack.
> And that's what I'm suggesting.  If we can create a universal buffering
> filter to accomodate such needs, let's not expect every author to reinvent.

I probably wasn't clear enough with this example.  If filters need to do
just buffering they should try to do coalescing too.  They should use
ap_append_brigade to put two brigades together, and store that one brigade
in their context.  That's how buffering should be done.

The problem is that we are trying to add too much to a single filter.  The
filter as it was proposed seem to do coalescing and buffering.  Wait,
let's define some terms so we are all on the same page:

buffering:  Storing lots of data in one place for use later.  It does NOT
matter what the memory looks like, as long as it is all referenc-able. For
example, a brigade is a type of buffer, because it can have lots of
buckets in it.

coalescing:  Reducing the number of buckets in a single brigade by
combining them.

The buffering bit is easy.  All we have to do is modify
ap_save_data_to_filter to call setaside for each bucket, and append one
brigade to another (already does all that), and then return the new
brigade.  The filter can then store that buffer in it's ctx pointer.

Coalescing is what I _think_ we are talking about here.  I don't want to
see any coalescing done except for in two places.  1)  in ap_r* as an
optimization, and 2) in the core filter just before we send.  In both
places this is an optimization.  The first place (ap_r*) reduces the
number of calls down the filter stack.  The second (the core
filter) reduces the number of packets that we send to the network.  

If we want to buffer data, do that with a call to
ap_save_data_to_filter.  If we want to coalesce, don't, that's the core's
job.  :-)


Ryan Bloom               
406 29th St.
San Francisco, CA 94131

View raw message