httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Kew <n...@webthing.com>
Subject Re: The Byterange filter -- a new design -- feel free to rip it to shreds
Date Mon, 12 Jul 2004 13:29:23 GMT
On Mon, 12 Jul 2004, Graham Leggett wrote:

> > at the moment when a byterange request goes to a dynamic module, the
> > dynamic module can not use any tricks to only serve the bytes requested,
> > it *HAS* to serve the entire content up as buckets.
>
> In theory, if mod_proxy (for example) gets a byte range request, it
> should only serve that byte range - ideally modules/filters should not
> "prop up" other modules/filters.

That will not always be practicable.  mod_proxy should be configurable
to pass byteranges headers straight through to the backend or strip them
and assume the proxy will handle the ranges.

> If a filter somewhere in the filter stack is going to break the byte
> range request in any way (for example something like mod_include) then
> that filter should be responsible for removing the Range header from the
> request before mod_proxy gets a chance to service the request.

Doesn't that break modularity rather badly?  mod_include is concerned with
simple content modifications, not HTTP.  It doesn't need more complexity.

> In theory, as the byte range filter should be one of the topmost filters
> run, it would have seen the Range header and noted what range it should
> have been returning, so a downstream filter removing the Range header
> should not cause a problem for the byte range filter.

But if you adopt that approach, then *every* filter has to faff about
with range headers (just in case), the first one strips it out, and
the others run in blissful ignorance.  Makes more sense if only the
byterange filter concerns itself with the header.

> In turn, if a downstream filter/content handler has returned a 206
> Partial Content response, the byte range filter should know what to do
> (has my job already been done by a downstream filter?

Yes, quietly remove itself from the chain.

> In fact thinking about this some more - mod_include might look at the
> byte range, and then intelligently decide to either include / not
> include certain included content based on the byte range. This could
> improve performance on some sites.

For mod_include to do that is an order of magnitude extra complexity
(even if you solve the problem of measuring the length of each include
without actually executing it).  For modules that generate entirely new
data - such as those based on a markup processor (accessibility, xmlns,
xinclude/xslt, proxy_html, annot - to name but a few) it becomes even
bigger: we'd have to count every byte we write!

> So to sum up:
>
> - Teach the byte range filter that it might receive content from a
> content handler that already has the byte range applied, and to react
> intelligently when this happens. A content handler will indicate this by
> returning a 206 Partial Content and/or a Content-Range header, which is
> easily parsed by the byte range filter - no need for special flags or
> buckets.

That has to be configurable, as some filters can only run on a
complete datastream.

> - Teach certain content handlers (such as mod_proxy or mod_cache) to
> handle byte range requests themselves, using the standard RFC2616
> headers and responses to indicate whether ranges have been applied.
> Which content handlers will be taught this will depend on whether there
> is a performance gain to be had by getting the content handler to know
> about byte ranges.

mod_proxy needs only two modes: transparent (leave it to the backend)
or opaque (get the entire document and leave it to the byteranges filter).
The latter would be appropriate when cacheing and/or content-filtering.

mod_cache in quick-handler mode is a special case.  But since that's
only serving from a complete document in-memory or on-disc, it's
straightforward.

> - Teach certain problem modules (mod_gzip if appropriate) to react

That'll be mod_byteranges.

-- 
Nick Kew

Mime
View raw message