httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Stein <>
Subject challenge requirements (was: summary: issues for my filtering patch)
Date Wed, 28 Jun 2000 21:44:26 GMT
On Wed, Jun 28, 2000 at 02:11:11PM -0700, wrote:
> I would much prefer an actual module that actually compiles and caches the
> response.  If it doesn't actually change the headers that are sent, but
> does make the ap_set_header call, then I will be fine with that.

I'll see what I can do. If it starts to add too much to the current patch,
then I'll come back. Again: I'd like to keep the original patch slim for
wider reviewability, your requested module as a future change, and the
support for that module another future change.

IOW, is it fair to state that the current patch can go in *without* this
particular module/support structure? That simply showing that your challenge
is possible, will validate that the current patch can support it?

> The bucket brigade scheme does not force ap_rwrite(), ap_rputs to allocate
> any memory, but for the time being, let's ignore that.

Any scheme does since the char* passed to ap_rwrite() has an unknown
lifetime. It *must* be copied *if* you want to change its lifetime.

The bucket brigade can say "here is a string, it will die when you return."
Your request here is to *also* allow the framework to say "here is a string,
keep it if you'd like... it will survive until you say otherwise."


> The criteria:

This is just a quick read through. I may come back when I dig in.

> 1)  The entire response MUST be help by the module at one time.  I don't
> care how many calls the module makes to actually write out the data, but
> the whole response must be accessable by the filter at one time.  This
> must hold true whether the request is 500 bytes or 500 MB.  (I think it
> goes without saying that if your module tries to keep 500 MB in memory at
> one time, the module doesn't meet my requirements.  :-)

No problem.

> 2)  No memory can be allocated out of r->pool.  If it is allocated out of
> r->pool, then the memory must be allocated for the length of the
> request.  Sub-pools are okay, but I will challenge them, and I will want a
> very good piece of psuedo-code to explain why the sub-pool won't require
> copying.


Note that r->pool and/or a subpool may be used to make a one-time allocation
of a fixed-size value (i.e. not proportional to the response size). This
should be okay. Agree?

> 3)  It is assumed that the filter is the first filter in a chain of
> filters.  ANY of the subsequent filters may also need to modify
> headers.  ANY of the subsequent filters may be either bucket or char *
> type filters.

Agreed. I had assumed that.

Note that the subsequent filters are somewhat moot since this first filter
is going to be holding back all the content until it is ready. When it
finally releases it, then the next guys can do what they will.

> 4)  It would be nice although not required if this module was written
> twice, once as a bucket type and once as a char *.  Because the two
> filters have different purposes, this may not feasable.  I can understand
> this.

The char* does not support this. You would not use a char* style of filter
if you need to hold back the entire response.

> 5)  The filter doesn't have to actually modify anything, so the module
> itself cannot allocate any memory to hold strings, except for structures
> used to store the strings.

See (2). Some fixed amount of memory may be allocated.

> 6)  The module cannot copy ANY data that it doesn't not
> modify.  Basically, what I am trying to protect against here, is having
> ap_rwrite copy the data into a buffer, and then the module copies it off
> to the side so that ap_rwrite can use the same buffer for the next
> chunk.  It is relatively easy to prove that this design requires that each
> modules copy all of the data.

Agreed. No worries: the module won't copy data.

> > If I provide the structures and detailed pseudo-code, will that be
> > sufficient? I can do this today.
> The more actual compile-able code there is, the better off we are.  One of
> the arguments you have been making is that your patch can do 99% of this
> today, as is.

Um. Then I was unclear. If I used the word "can", then I meant it in the
future tense (i.e. "it can do that [with changes]"). Let's rephrase as "my
patch could do this with straight-forward extensions."

Your "challenge" is in response to that statement :-). Fair enough.

> I can understand that the headers that are actually sent to
> the client will not be modified.  I can understand that it will not be
> optimized.  If there are structures that are required but not implemented
> yet, the more detail there is the better.

No problem.

> In the end, my conditions may not be explained in enough detail.  I have
> tried to think of everything, but I probably missed something.

That's all right. I understand the problem space ("hold the entire response,
modify a header, release the response; zero copying occurs within the
filter"). I'll write code to solve *that* problem. If that doesn't agree
with you, then we can iterate.

> I will of course try to be fair when reading through the module, and I
> will try not to make up new requirements, but it is VERY likely that I
> will apply the current requirements in strange ways.  If I do, I will
> endevour to explain why I have done so.  If new requirements are brought
> up during the review, I will explain exactly why they are issues.

Fair enough.

> I believe I see how to write a module that meets all of these requirements
> with the bucket brigades design.

I do, too. I also see how to do it with my current patch. :-)

> This will just remove my veto.  I will not give this patch a +1, I will
> give it a +0, and step out of the way.  I think that is fair.  Greg, this
> is the best that I can do.

Absolutely fair. Thanx.


Greg Stein,

View raw message