httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject RE: summary: issues for my filtering patch
Date Wed, 28 Jun 2000 19:43:53 GMT
> > Exactly!  The filter can't avoid it.  So, then the question is how does
> > the filter buffer the data.  There are two answers to this:
> > 
> > 1)  The filter just comes up with it's own method for filtering, and if
> > that means keeping the whole request in memory, then so be it.
> If this happens, and no filter -wants- to manipulate the output, the
> CGI's behavior may be async, and we can start showing users the results
> of their inquiry immediately (or at least the table headers for data
> to come in 15 seconds or so.)

This will happen in any approach.  Neither approach puts a limit on how
little memory is passed to the next layer.  However, I believe the buckets
approach does encourage filter writers to send more data down to the next
filter at once.  This is a gut feeling, and is not proven, so it's really
not worth arguing.

> > 2)  The core imposes some sane restrictions on how much of the request is
> > in memory by controlling the buffering.
> Then it sounds like we can't see async behavior?  Unless some filter really
> -must- hold the headers until the entire data stream is cached, then this
> is sounds like a very bad behavior.  Who -must- hold headers until the
> entire body is constructed?  Perhaps my own mod_robots, and it only cares
> about reparsing the <HTML><HEAD>[<META ...>...]</HEAD> stream,
so even it 
> should release the request fairly quickly.  

ANY module that wants or needs to modify a header MUST cache the entire
request until it knows that it is done modifying headers.  This means that
I believe mod_auth_digest will need to cache the entire request, because
it needs to modify a header to have the md5sum.  Any module that needs to
modify the content length needs to cache the entire request.

> Few filters should even need to do what this suggests.  I agree the server 
> aught to become responsible and prohibit really bad memory behavior by 
> laying out the rules and mechanisims, but I strongly believe either patch
> can and should provide the api to address this issue.  But the abuser of
> this feature should be the one to jump through hoops, not the rest of the
> module writing world :)

This is why I have asked for the module.  When I see that the current
patch can allow modules to cache the entire request without copying the
entire request (I don't really even mind if the module keeps the whole
thing in memory, although a comment explaining how it would flush to disk 
would be nice), I will remove my veto.  I personally do not believe this
can be done.

I see how this module would be written with the bucket brigade scheme.  I
do not see it with the current patch.


Ryan Bloom               
406 29th St.
San Francisco, CA 94131

View raw message