httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Life is hard, and then you die" <>
Subject Re: recap of filtered I/O
Date Sun, 04 Jun 2000 22:38:10 GMT

Note: the linking of filter/layers via hook's or linked lists is really
independent of the underlying scheme, hence I will use the term filters
for Ryans filtering proposal, and layers for Greg's proposal.

On Sat, Jun 03, 2000 at 02:26:16PM -0700, Greg Stein wrote:
> On Sat, 3 Jun 2000 wrote:
> >...
> > Yep, your making perfect sense.  I hope to have a new patch next week
> > sometime that addresses all of Greg's points and still uses the hook
> > mechanism.  :-)
> My current concerns with the hook mechanism:
> *) complexity for the module programmer due to iovec[] strategem
> *) how would mod_include.c::include_cgi() be written in a hook scheme?
> *) how would my insert-100Mb-filter be written?

I Ryan addressed this point with the filter being able to signal that
it hasn't sent all yet.

> *) if the filter wants to insert a file, then how do we get the FD
>    returned to Apache so that it can use sendfile/TransmitFile?

This is something that's not clear to me in *either* scheme. Can you
elaborate how this should be done using layers?

> *) flow control

I don't see this as really any different from the layer approach. But
I'm assuming that a filter/content-generator that creates large output
will do so in parts (third point above), and only smaller stuff will
slurp all up and send it in one write.

> *) how would "SetFilter SSI PHP" be implemented in the hook scheme?
>    (note this implies a table mapping names to functions; also, I believe
>     that solving this directive for the hook-based scheme will essentially
>     look just like the link-based scheme)

I think this is a minor point. The hook scheme allows you to order a specific
filter after another one. Yes, it essentially degrades to a linked-list.

> *) hook-based scheme potentially has a larger working set because the
>    iovec[] return values must occur on the heap
>    (in the link-based scheme, they may occur on the heap or temporarily on
>     the stack)

Let me say that on the whole I prefer the layering scheme Greg is pushing,
because that's what I'm used to ;-)  I find it very convenient to write
layers that way. Trying to assemble iovec's seems somewhat cumbersome, and
I haven't seen any advantage of iovec's yet which would make it worth it,
unless it's that it may be more efficient in the end on systems that
support iovec's on sockets.

OTOH, I've been thinking how this would integrate with Dean's proposal to
have a dedicated process/thread which serves slow clients (via some
select/poll based pseudo-asynch-io) so as to free up the processes/threads
for doing the content-generation/filtering work as quickly as possible,
which I think is a really cool idea.  With the iovec's it seems more
straightforward at first glance, because it would get all the data it
needs in one go. However, as soon as the filters can produce partial
output this reverts to what I'm guessing will be needed for the layer
scheme: a largish buffer into which the stuff is assembled (i.e copied),
which the slow-client-process can then dribble to the client at will. So
I'm not sure I see any advantage here using iovec's either. However, I
suspect I may be missing something, because I also don't see how sendfile
integrates into these schemes - I'm starting to think Roy's
bucket-brigades may be the way to go after all.



View raw message