httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: filtering patches
Date Mon, 10 Jul 2000 14:56:07 GMT

I am not going to argue this too much.

Yes, the API's change.  In order to do filtering correctly they have to
change.  Filters need more control of the data then just sending out
streams of data.  If we start with your patch, we will end up with this
patch at some point.  I say that, because your patch will need all of the
new bucket types, as well as all of the functions to combine the bucket
types.  Then, we will need ways to split a single bucket into multiple
buckets.  We are doing more complicated things than just writing
data.  The API is going to get more complex.

As far as their being bugs, I admit that completely.  I am working to fix
the biggest bugs.

Pools will not work for filters.  The reason is simple.  They don't free
memory.  If I try to write 100MB and I allocate it from a pool, that 100MB
is allocated for the length of the request.  It can't shrink until I am
done with the request.  Pools will work if we play games with sub-pools,
but that requires that no modules destroy the request->pool or any of it's
sub_pools.  There is one way to free a bucket, it is through the _destroy
function.  If it is not destroyed, we still have the data.  Using pools is
going to end up meaning that no filters or generators will destroy_pools
for fear of ruining somebody else's data later in the chain.  

I will be fixing things, but the direction of both patches is clear.  Both
patches have bugs, but those can be fixed once one patch is
committed.  It's the direction that is important at this point, not the
actual patch.


On Mon, 10 Jul 2000, Greg Stein wrote:

> I've just checked in an update to STATUS noting a veto on the two patches
> that Ryan has posted. I'm not going to go through a serious, in-depth review
> of the patches here; instead, I'll just hit on the high points:
> *) all content generators (Apache's and third party's) must be rewritten to
>    take advantage of this filtering mechanism
> *) these patches do not use BUFF, but attempt to go straight to the network.
>    this loses the buffering, the chunking transfer coding, and the character
>    set translation which BUFF provides.
> *) the use of "rwmem" is unclear; it appears to have a fixed size buffer and
>    requires that all generated content be copied into a holding buffer. if a
>    content-generator creates 100Mb of content, can it do this piecemeal
>    (create/destroy bucket brigades for pieces of the content) or will it get
>    spooled to disk (per a comment in ap_rwmem_write)
> *) minor issues:
>    - the use of malloc() is scary; I do not understand why a pool is not
>      used (scoped to the request, the connection, or a cache)
>    - bugs, such as "newbuf = malloc(sizeof(newbuf))". the server doesn't
>      even operate if these patches are applied (not withstanding the BUFF
>      avoidance mentioned above)
>    - much more complex API (at least a couple dozen new functions)
> ------------------------------
> Conversely:
> *) the patches that I've posted require no changes on the part of today's
>    content generators, nor will it be required in the future.
> *) The patches continue to work with BUFF, which has been (effectively)
>    isolated into a filter; a future pass can turn it into a real filter, can
>    shift translation out (e.g. the mod_xlate that I posted), and can shift
>    chunking out (thus leaving it to simply buffer content for the network).
> *) the output mechanism is the same as today: ap_rwrite() and friends.
>    Filters use ap_fc_write() and friends -- analogues to ap_r*().
> *) no known bugs. the server operates quite fine, and I've tested the
>    patches with several actual filters, performing different types of
>    filtering.
> ------------------------------
> I think it is time that we get some additional commentary and review of the
> patch sets from the other project members. I know that my patch is "final"
> for this pass; Ryan will probably resubmit with some bug fixes.
> The STATUS file has been updated with Message-ID values for the location of
> the different patches, and the mod_xlate demo that I posted. Of course, it
> also has the current record of voting for the three patch sets.
> Please take some time to review these patch sets. Thanx!
> Cheers,
> -g
> -- 
> Greg Stein,

Ryan Bloom               
406 29th St.
San Francisco, CA 94131

View raw message