httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: code chunks for filter challenge
Date Thu, 29 Jun 2000 22:56:18 GMT

> > But isn't that information lost when we go from bucket to char * filters?
> Yes, it is lost.
> There is no sense in continuing this email response if you focus on the
> char* filters.

I've been focusing on the char * filters since the beginning.  This is
where my issue is with your design.

> You asked for a mechanism to allow zero-copy from the content generator down
> to the network, as it passed through the filters, and where one or more of
> those filters must set ALL the data aside. Great. I coded it.

Yes, you did.  I have no problem with what you added last
night.  Unfortunately, either you haven't been listening, or I haven't
been clear enough.  The char * filters mean that EVERY filter must copy
the data.  You said this wasn't the case, so I asked you to prove
it.  What you proved, was that the char * filters defeat the purpose of
the bucket filters.

> My mechanism has another useful feature which allows a person to write a
> char* filter. This is INDEPENDENT of the requirement at hand. If people want
> to insert a char* filter in there, then fine. We also allow people to write
> infinite loops in their modules. We cannot be the police for everything.

This isn't independent of the requirements at hand.  I tried to highlight
this yesterday, when I was talking about the sebsequent filters on the
stack.  If the design forces a memcopy (as the char * filters do), then
the design is broken.

I don't think I can express this any clearer.  I said this yesterday
morning in my discussion with Jon Winstead, and I _THINK_ he understands
what I am saying now.

> You wanted proof that my filtering mechanism can support a zero-copy with a
> complete set-aside of the response. I provided that. I do acknowledge that
> the filter writers must use buckets throughout to see this happen.

More than that.  EVERY filter writer must use buckets to see that this
happens.  This means that if ANY filter in the filter chain uses char *,
then the benefits of the buckets disappear, and in fact we take a hit by
using them.

> Look at it this way: assume that the char* filter does not exist. Review the
> set-aside code in that light.

You can't do that.  The char * filters are the basis for the design.  Next
thing you'll be telling me that we can just remove the char * filters

> [ take the position the char* feature is added later ]

I am against the char * all together.

> I spent a long night meeting your challenge. The code *can* do what you want.

Yes, it can.  If we remove the char * filters.  With the char * filters,
it cannot.  This is what I have said over and over again.
> Please review the code as if a char* handler does not exist. Lift your veto
> if you find no problems in that light; explain your issues otherwise. If you
> would rather shortcut an additional explanation, and can agree that char*
> handlers are not an issue, then I'll review your challenge response in
> detail, explaining why certain allocations and copies exist, how the
> lifetimes work, etc. Those items are all to be expected given the current
> Apache APIs for content generators to write data.

The char * filters are the ONLY issue.  They have been the issue since
almost day one.  They have especially been the issue since early this week
when Roy's bucket brigades finally made sense.

The current API's for content generators do not put these limits on us.  I
do have a design that I will implement soon which does not have ANY of the
limitations of this patch.  The design is what I believe Roy is talking
about with his bucket brigades.

Everything that you admitted to above is what I dislike about the design,
and it is why I have the veto in place.  In previous e-mails you have
certainly made it sound like non of this was true:

>> >               For the simple callback, the pool handling occurs
>> >           the callback. It remains just as simple as above. The point
>> >           here is that we have optimization choices that will not
>> >           the simple_cb style of filter.
>> The point is that this looks like an optimization, but it is really not
>> one.  In reality, this design (as well as any other that requires
>> pools) requires more memory and pool memcpy's than are necessary.
>False. See above.
>The framework retains no additional memory to deal with the char*
>As always, the filter is free to consume memory, but there is nothing in
>design that encourages unlimited consumption.
>A memcpy() is only needed if data must be set aside. This occurs in all

A memcpy is required once a filter uses a bucket filter and the next
filter uses a char *.  That's what the code you posted yesterday did.

> 4) the char* handler is bad for various reasons
>    => I showed that each of these "reasons" are issues for filters in
>       general and not characteristic of the char* handler. regardless,
>       the
>       "reasons" do not preclude the patch, the design, or filters in
>       general.

You admit that this is so at the top of this e-mail.

> 8) need to track pools and their lifetimes
>    => false. demonstrated otherwise

Doesn't the code you designed last night track pool lifetimes to make sure
we aren't trying to keep memory around to long.  I could have sworn
that's what ap_lsetaside did.

> 9) the framework demands extra memory and memcpys
>    => false. demonstrated otherwise

Anytime char * filters are used, we have extra memory usage and extra
memcpy's, you admit this above.  This is why the char * filters are bad.

> 10) the framework precludes certain optimizations
>    => possibly (as with any code), but there has not been a credible
>       example of an optimization that is: important/relevant, and can't
>       be done later within this framework

We just proved that wrong.  The optimization that cannot be done in the
current design, is zero-copy.  As long as the char * filters are present
in the patch, this optimization is impossible.

My veto is remaining, because this design is flawed.


Ryan Bloom               
406 29th St.
San Francisco, CA 94131

View raw message