httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: BIG filtering patch.
Date Fri, 07 Jul 2000 15:29:54 GMT
On Thu, 6 Jul 2000, Roy T. Fielding wrote:

> >This patch uses hooks, but it is completely recursive, and I expect to add
> >filter names at some point.  
> What is the rationale for using hooks?  They just seem to be

There are a couple of rationals for using hooks, I will try to explain
here.  If it still doesn't make sense, please let me know.  :-)

> obfuscating what should be a simple stack of filters.  

A long time ago (at the beginning of this whole discussion) we decided
this shouldn't be a simple stack of filters.  The argument was put forth
that there are basically five types of filters, and there is an obvious
ordering for how those five types should be called.

The five types are:

content-generators     handled by the handler hook
content-filters        anything that munges data
content-encoders       anything that changes the representation of the
		       data without changing the meaning
content-processors     anything that changes the meta-data about the data
content-transporters   The stuff that actually transports the data.

The argument was made, and I still agree with it, that all content-filters
should be called before the encoders, which should be called before the
processors, which should be called before the transporters.

Using the hooks allows us to easily order the filters in these categories.

There are two other features of the hooks that I like:

The code is there, and it works.  And, module writers will already know
how the hooks work.

Adding a stack implementation into Apache just adds more code.  Our tables
can implement the same thing, with code that we know works.  Not that a
stack is hard to implement.  It's just more code that we don't really need

The easier we make it for module writers to add a filter, the more filters
there will be.  I was really hoping to not make module writers learn two
API's for inserting hooks into the server.  One for filters, and one for
everything else.

I can take the hooks stuff out, I'm not married to it, but I think it has
a lot of features we don't want to remove, like the automatic ordering of

> We don't
> want filters to be self-sorting in any case, since that would
> interfere with layered encodings and subrequests.  

I guess I don't see this, but I am willing to be educated.  In my mind, it
all comes back to when filters are inserted onto the stack.  I guess I
look at it and think it isn't valid to insert a filter that is going to
munge the data once we are encoding the data.  This means that by the time
we are done munging data, all of the mungers have been added to the stack
and we can forget about them.  The same thing goes for all of the rest of
the types of filters.  I see the self-sorting as an issue in two cases:

1)  We try to add a content-filter while we are encoding the data.  This
is invalid in my mind (although I could be wrong) and not something to
worry about.

2)  We add a content filter while we are filtering the data.  In my head,
the function that is inserting the filter has to know enough to put the
new filter into the list after the current filter.  The sorting algorithm
shouldn't touch it, because when we inserted it into the list, we gave it
the right number.

Having said all of that.  I kept that section of my latest patch as clean
as possible for a reason.  If people really want to remove the hooks
stuff, it should be relatively easy to do.  We need a way for filters to
easily insert their filters, and those filters need to be
order-able.  Greg's patch relied on the insert-filter hook to do the
ordering.  My patch asks the filters to order themselves.  I haven't given
any thought to how a stack would work, but my gut feeling is that it
wouldn't be great, simply because we take the chance of asking mod_include
to run after we have gzip'ed the data.

Does that make sense?

> It also doesn't
> serve our purposes to be calling a core register function to insert
> filters when all we ever want to do is push them onto a stack and
> pop them off a stack.

I assume you are talking about the core_insert_filters.  This is just for
the core to insert it's filters.  All of the patches currently have a
function that allows modules to insert their filters, whether they are
pushed onto a stack, added through the hooks, or added to a linked list,
the modules need to have a place to insert the filters.  The very first
patch I made didn't have this feature, and modules just inserted filters
whenever they saw there filters were necessary.  People didn't like that

I expect that over time we will find that we really need three or four
functions to insert filters on the stack.  I know I want a function in
mod_cgi[d] so that we can allow mod_php, mod_perl, mod_include, etc to see
the output of a cgi script.  I don't think we want mod_cgi[d] to be
responsible for actually inserting the filter onto the stack.  We just
want mod_cgi to tell the other modules that they can insert their own

> I think if we can agree on an answer to that one question then it will
> be relatively easy to merge all of our stuff into a single sandbox and
> jump start a coding frenzy.

I assume you are talking about the filter stack from Greg's patch, and the
bucket's from my latest patch.  I can do that merge next week if nobody
else beats me to it.

> The usage of ap_bucket_brigade_to_iol seems a bit odd as well, since
> an iol itself should just be a filter, but that can wait.

That function is there to let us flush the data to the disk or to the
network.  Originally, I had called it ap_bucket_brigade_to_disk, which
opened a file, and flushed the whole brigade to the disk.  Then I realized
that basically the same code would be needed to write out to the network,
so I changed the name.  :-)  We can easily make iol's a filter later, but
we will still need a function to flush the data out of memory and to the
disk.  That's what this function is meant to do.



Ryan Bloom               
406 29th St.
San Francisco, CA 94131

View raw message