httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: summary: issues for my filtering patch
Date Wed, 28 Jun 2000 17:00:09 GMT

> > 2)  The core imposes some sane restrictions on how much of the request is
> > in memory by controlling the buffering.
> > 
> > Greg is proposing #1, I am proposing #2 (after much prompting and head
> > beating from Roy).  My feeling, after having looked at this problem for a
> > VERY long time, is that trying to just fit option 2 in after the current
> > patch is applied is almost impossible.
> What are the restrictions imposed by the core in this scheme?

Basically, the core will provide buckets, which can get so big.  If the
buckets get too big, the buckets themselves flush out to the disk, and

> > Using option #1 is not a good idea IMNSHO, because it allows stupid
> > filters to take down the server.
> I don't see how you're ever going to prevent that. People always
> find ways to do stupid things. :)

Allowing people to hang themselves, and not providing a way to prevent
them from hanging themselves are two VERY different things.  Yes, if
somebody wants to write a filter that takes down the server, then they
will.  If we don't provide a sane bucket scheme, then ANYBODY who writes a
filter has a very good chance of taking down the server.

> > By having a top level filter do the buffering it requires, we can actually
> > get single or zero copy in.
> It sounds to me like that is resolved just by defining the memory
> life-cycle of a "bucket" to always end once it is written to the
> network.

Defining the life cycle of the bucket to end when the bucket is sent to
the network doesn't have anything to do with single or zero copy.  The
problem is that in the current patch, no information about the memory is
passed with the memory.  This means that each filter must assume the
memory cannot be touched.

By passing information about the memory around with the memory, filters
can determine whether they can modify the data in place, or if they have
to copy it over in order to modify it.

In the current patch, any module that wants to modify the data must
allocate more memory (whether that be on the stack or heap it must be
alloacted).  This is because you just don't know where the memory came
from, or what somebody else is doing to it.

> If some intervening layer needs to make a copy (to store for later
> use, or to move it to an area it can change), I don't see how you
> can avoid that.

What you can avoid is the _need_ to move it to an area it can
change.  Whether the data is modify-able is information that can be passed
around with the memory.  Take a look at Roy's bucket brigades post, and
Dean's original implementation of ap_buf.[ch].  When I really started
reading those two things, everything seemed to come together for me.

> I'm a little confused as to what you mean by "top level filter".
> Is that closest to the network, or some distance away?

I consider the filters to be a stack.  The top level is the first filter
the data passes through.  the bottom filter is the one that writes out to
the network.

Basically, what I was getting at was that if the first filter that needs a
copy of the data makes a copy and marks it as modify-able memory, then no
other filter in the stack needs to make a copy, because it can just modify
it in place.

> > Again, it is how the buffering is done that is important.  Asking each
> > module to implement their own filtering is asking for trouble.
> Some modules (like PHP) are probably going to implement their own
> buffering (which is what I assume you meant to say) regardless of
> what the core does or provides. It doesn't seem like a good reason
> to hold back filtering to me.

PHP can add it's own buffering in, there is nothing to stop that and there
is no reason to.  What we want to keep from happening, is to make EVERY
SINGLE filter implement their own buffering.


Ryan Bloom               
406 29th St.
San Francisco, CA 94131

View raw message