httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tony Finch <>
Subject Re: subrequest filtering (yet again)
Date Thu, 21 Sep 2000 19:22:34 GMT wrote:
>On Thu, 21 Sep 2000, Jim Winstead wrote:
>> i don't see how including the results of a subrequest should be
>> any different than including a file. or the results of an external
>> request. or any other bit of data that a content-generator generates
>> (whether it be the first thing in the filter-chain or something in
>> the middle that is inserting content).
>> it almost seems like it should be its own bucket-type.
>This is actually one of the designs that Tony and I considered, but it is
>a bit non-orthogonal.  The problem is that sub-requests basically convert
>to some other bucket, be it a file or heap, whatever.  It is not possible
>to read from a sub-request bucket until it has been converted.  This
>conversion could be hidden behind the read function, but it isn't
>completely clean.
>Imagine a sub-request that resovles to a CGI script that takes a long time
>to output all of it's data.  The sub-request bucket has to stick around
>throughout all of that processing.

It's a bit nastier than that.

At the moment the way we get data hold of the data in a bucket is via
the read() method, which may have the side-effect of reading the data
into memory if it isn't already there. The idea is that we delay this
operation as long as possible so that if the filter chain is really
simple then buckets can get right down to the core filter without
mutating so that the core can handle them in the most efficient manner
it knows (e.g. sendfile).

URI buckets can use this API, but that would compromise efficiency --
sendfile would then be unusable because file buckets would mutate into
mmap or heap buckets in order to bring the file data into memory.

The alternative is to add a new API that unwraps the URI bucket into a
bucket brigade representing its contents in the most efficient manner
possible. This is a disaster if the sub-request is for a CGI
outputting SSI, because there is no efficient way to represent the
output: the entire output must be buffered in memory. (The
non-orthogonality arises here because this operation makes very little
sense at all for the other types of bucket. Yes, the same is true for
setaside() and I think it is bogus too.)

So what is required is some compromise between these two designs that
maintains the efficiency advantages of buckets whilst still being able
to process the output of the sub-request as it is produced if this
happens to be a long slow job. We already have such a design for
handling the response data from the main request, so it makes sense to
use that.

en oeccget g mtcaa    f.a.n.finch
v spdlkishrhtewe y
eatp o v eiti i d.

View raw message