httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: cvs commit: apache-2.0/src/main http_protocol.c
Date Thu, 12 Oct 2000 23:57:27 GMT

> > The solution is to use an integer pointer instead of an integer in the
> > filter function argument list.  Basically, on input we are specifying the
> > maximum amount that we will allow to be READ, not returned, but read.  If
> > the filter changes the data, then the length has to grow, and the filter
> > has to inform the caller of this.
> > 
> > This is not how the code is written currently, but it is a simple change
> > if people agree.
> Um. I disagree.
> 1) I don't understand your explanation about read vs return.
> 2) When the filter is called, the caller is *only* interested in what comes
>    back to it. Whatever the filter does underneath, that is the filter's
>    problem.
>    [ if http_filter read past the end of the request, then it better pretend
>      that it didn't, and save the data for the next request for later ]
> I'm not sure what Jeff means about "can't have filters that increase the
> length." Sure we can. Consider a gunzip filter. Somebody calls into the
> filter saying "give me 1024 bytes of data". gunzip calls to the next one and
> says "give me however much you want, I can take it all." It then unzips
> whatever is returned. It returns (up to) 1024 bytes and saves any remainder
> off to the side, for a later call into the gunzip filter. Only when that
> saved-to-the-side bit runs out, does it call down for more.

All of the above is exactly what I thought originally as well.  In fact,
this influenced the design we have.  However, the problem is how are the
input filters used.

What I have come to realize, is that my goals for input filtering and the
goals Jeff and Greg have are very different.

I'll be working backwards, because I think it will make more sense.  Take
the gunzip filter.  The content length tells Apache when to stop reading
data from the network.  Greg, in what you said above, we end up losing
data.  It is a sneaky loss, but it exists.  Think about it this way.  We
start with a content length of 25.  This is 25 bytes of gzip'ed data that
actually represents 35 bytes.  We start by calling down through
gunzip_filter to the http_filter, saying give back at most 25 bytes.  The
http_filter gives 25 bytes to the gunzip filter, which uncompresses to 35
bytes.  Now the gunzip filter is only allowed to return 25 bytes, leaving
10 that are saved off to the side.  The problem is that as far as Apache
is concerned, we have already read all the data from the network, and
there is no reason to call down to the http_filter again.  If we do
in fact, we will have read past the end of this request, because we called
http_filter when we weren't supposed to.

This gets more annoying if we originally read in 15 bytes, and those get
uncompressed to 25.  Then we just ignore the other 10 bytes that are on
the network.

This is also the difference between read and return.  The amount of data
read is the amount actually read from the network.  The amount returned is
the amount that is returned from the previous filter.  Again, this is
defined by the bucket brigade, so we can remove that parameter.

What else does this mean?  It means that all of a sudden
ap_get_client_block can actually receive more data from the filters than
it asked for.  The filters can't read more from the network than they were
told to, but they can transform the data into a larger chunk of data.  So,
we will need to be able to save the extra data from the filters.  To do
this, we don't want to use the conn_rec, because this is request data, so
we will need a brigade in the request_rec.

I dislike this, and there is a big part of me that thinks we are violating
the HTTP protocol by allowing filters to expand the body data like
this.  The other thing that needs to be noted, is that if we change the
content-length, then that information also needs to be changed in the


Ryan Bloom               
406 29th St.
San Francisco, CA 94131

View raw message