httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject RE: cvs commit: apr-util/buckets apr_buckets_file.c (fwd)
Date Fri, 16 Feb 2001 18:02:49 GMT
On Fri, 16 Feb 2001, Cliff Woolley wrote:

> > -----Original Message-----
> > The second problem is one I haven't fixed however.  Brian, if you increase
> > the size of your file to 10Meg, then mod_include will read all 10Meg into
> > memory before sending it down the stack.  That's BAD!  mod_include needs
> > to be taught how to stream data when there are no SSI tags in the file.
> >
> > I will be adding the second issue to STATUS.
> The more I think about it, the more I like the idea I mentioned to you
> sideband of teaching mod_include that before it reads from a file bucket, it
> should copy it; if it gets a heap bucket back from the read but finds
> nothing in it, it can destroy the heap bucket, manipulate the stored copy of
> the file bucket, and reinject it back into the brigade.  That way we'd only
> have one 8K chunk in memory at a time for inspection; other 8K chunks would
> remain in memory only if they have SSI tags in them.  Sound good?  I'll work
> on a patch for this this afternoon.

My only problem with this, is that this would be mod_include specific.  I
have thought of something like this before, but it always seemed like an
almost impossible thing to do with any degree of generability.  This also
means that we would still be parsing the whole file before we pass it down
the filter stack.  I really don't think we want to do that.  I realize
that just looping through a 50K file doesn't take long, but what happens
when we have to loop through a 100Meg PDF file?  We want to stream that.

Also, this fix is very specific, it only works for files.  What happens
when we have a CGI that outputs 10Meg of data to the include filter, but
no SSI tags?  We end up with 10Meg in memory again.

I also have to wonder how useful that feature is.  We definately don't
want to go from file to mmap, back to file.  So, this feature is really
only useful on platforms without MMAP.

Personally, I would like to see mod_include stream the data.  However,
feel free to implement what you have described, it does have some
interesting properties, namely that we can actually use sendfile on a file
that has gone through mod_include as long as there were no SSI tags.


Ryan Bloom               
406 29th St.
San Francisco, CA 94131

View raw message