httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: Filtered I/O ... again. :-)
Date Wed, 31 May 2000 19:39:05 GMT

> > This is not painful.  And ordering isn't always necessary.  If I have a
> > simple filter that just adds plain text to the top or bottom of a
> > page.  It doesn't matter if that filter is first, last, or in the
> > middle.
> Sure it does. I've got my filter that rewrites HREF values to include a
> "cookie" to deal with the user's non-cookie-enabled browser. If your
> filter runs first, then I get to rewrite the HREFs. If your filter runs
> after, then I cannot properly do the rewrite.

Huh?????  My filter adds "plain text" perhaps I should have specified that
I meant without HTML tags of any kind.

> Another case: your filter ran before my PHP script. Oh crap, it sent out
> an HTML header, so my PHP script can no longer set the HTTP headers.
> Again: I have to enforce that your "simple filter" runs *after* my PHP
> filter.

Your PHP script is not a filter, at least I wouldn't expect it to
be.  Your PHP script should be a generator.  My filter simply gets a
chance to look at everything your generator writes out.  My filter CAN'T
run before a PHP script unless PHP is re-implemented as a filter.  In
which case, My filter didn't send anything out over the network until the
PHP filter has had a chance to review and/or modify it.  I do not follow
this case at all.

> I can come up with more. :-)

You still haven't come up with one  :-)

> > > Definitely an allocation, and there is no way to avoid it in your
> > > proposal. (that I can see)
> > 
> > Try some pointer manipulation.  The only allocation that is required here,
> > is the iovec[1],
> Agreed. I said "an allocation". Technically, there might be two, when you
> build a larger iovec, but we can easily pad the iovecs out to N. The
> insane filter could just deal :-)

But you imply that the link-based method doesn't have this allocation, and
it does.

> > which is unavoidable regardless of whose implementation
> > is used.  If you have some method of avoiding the allocation of iovec[1],
> > I would love to understand it.
> I was going to say "mmap", but I realize that an mmap would also work for
> the iovec case.
> BUT: the hook-based design requires that the iovec elements are
> *writeable*. Therefore, you could not simply mmap the file. You would have
> to copy it onto the heap.

This is just not true.  Why does it have to be writable?  Only the iovecs
that are changing need to be writable.  Take your original example:

---- foo.shtml ----
some crap
<!--#include mmaped_file.html -->
more crap

iovec[0] = "some crap"   ---  not writeable
iovec[1] = included file ---  writeable
iovec[2] = "more crap"   ---  not writeable

Just about every filter will be able to be broken down like this, where
parts will need to be writable regardless of the method used, and other
parts won't.

> Maybe you could expand the mechanism some more and tag each iovec as
> readonly or read/write. If a filter wants to modify (in place) a readonly
> value, then it will have to copy it first.

exactly, and this will have to be done using either link-based or
hook-based.  The only difference I can see is how the information is sent
along to the next filter.

> Okay, an alternative: ap_send_fd()  An allocation could be avoided by
> using ap_send_fd() to let the kernel shove the thing out the pipe. Of
> course, if there was another layer in the way, it would have to be mmap'd
> into memory so the next layer could deal with it.

So mark parts of the file as untouched, and ap_send_fd can still be
used.  This improvement can easily be added to the hook-based design.

> Essentially, I believe the linked-list design has more room for
> improvements and optimizations than the hook-based design. The send_fd
> thing is an example, but it (arguably) does not occur often.

I disagree that this can't be put into the hook-based design.

> No, there is no tail recursion. The first parameter to ap_lwrite() is
> effectively the "object", if you will. I could rewrite the thing as:
> Please go back to the post that I referred to. The implementation of
> ap_lwrite() is in there. You will see that it is *very* simple.

Rather than deal with your example, I am going to deal with the code you
posted before to prove their is either tail recursion in your design or I
am completely missing how you want to do this.

I have just looked at your code from that message, here is what your
function looks like:

*) ap_status_t ap_lwrite(ap_layer *layer, const void *buf,
                         size_t len, request_rec *r)
       if (layer == NULL) {
           ap_bwrite(r->connection->client, buf, len, &amt);
           return OK;
       return (*layer->func)(layer->next, buf, len, r);

Now, unless I am wrong, each filter calls ap_lwrite to pass the data down 
to the next filter.  So, we have two modules foo and bar which implement
foo_filter and bar_filter respectively.  (foo|bar)_filter each call
ap_lwrite (as implemented above) to pass their data to the next layer in
the list.  Let's say for example that the list is:

http_core  ->  foo_filter  ->  bar_filter  ->  ap_bwrite
(generator)    (filter)        (filter)        (write to network)

http_core calls ap_rwrite which calls 
    ap_lwrite(foo_filter->bar_filter, buf, len, r);
        foo_filter does some stuff to the data and calls
            ap_lwrite(bar_filter, buf, len, r);
                bar_filter does some stuff to the data and calls
                    ap_lwrite(NULL, buf, len, r);

Now, I see three calls to ap_lwrite, and we don't return from one until
the subsequent one is done.  In my book, this is indirect
recursion.  Since the last thing ap_lwrite does is make the call which
results in the recursion, this is tail recursion.  Put them together, and
I see indirect tail recursion.  Please explain what I am missing, because
I have looked at your code and your call stack, and I keep seeing tail
recursion in them.

> > Plus, I am using an abstraction that we already have and
> > understand.
> This is arguable. You are introducing: per-request hooks, a new hook
> semantic, a new way to insert that hook, and a new mechanism to implement
> a filter.

The hook semantic hasn't changed as far as module writers are
concerned.  What they know about using regular hooks is directly
applicable to request-based hooks.

> > Also, your method asks module writers to implement multiple
> > functions (at least that's how I read it), lwrite, lputc, lputs,
> > etc.  (one lfoo for each rfoo).  This is a huge hurdle for module
> > writers.  My design asks for one function that just works.  With a couple
> > of supporting functions (to be added when this is accepted), writing a
> > filtering module is trivial.
> Sorry. Please reread the proposal that I referenced.

*) Apache defines ap_lwrite(ap_layer *next_layer,
                            const void *buf, size_t len,
                            request_rec *r)
   and possibly some similar ones for printf, puts, etc

> Filter writers need to implement just *one* function (of type
> ap_layer_func_t). For their output, they use ap_lwrite(). Nothing more.

Then why exactly is Apache implementing ones for printf, puts, etc?  You've
just said nobody is going to use them?


Ryan Bloom               
406 29th St.
San Francisco, CA 94131

View raw message