httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Stein <>
Subject Re: filtered I/O - flow control
Date Thu, 01 Jun 2000 20:45:18 GMT
On Thu, 1 Jun 2000 wrote:
> > >           If at any point there is too much data to write to
> > > the network, it will be up to Apache's internal buffering to catch it and
> > > hold it until the socket is ready.  
> > 
> > Seems like that could potentially cause a large increase in storage use
> > for huge files.  
> There is no more potential for this than there is currently.  Remember,
> that in both approaches, the maximum that can be sent to the filter is the
> current maximum that can be sent to the network.  In both approaches the
> amount of data returned from the filters is likely to grow very large, but

The link-based approach does not "return" data. It sends it down through
the layers to network as it is being generated.

> in both approaches, the filter is going to end up being limited by the
> same limits that Apache has.  I believe this is a red herring.

Actually, I think that Greg has an incredibly valid point here. One that I
certainly didn't see, and one that makes me even happier with the
link-based approach :-)

> > If I understand how it works today correctly, we typically have only 2
> > moderately sized BUFF buffers tied up for output - one for sockets I/O,
> > and one for either file I/O or CGI output.  The process/thread ends up
> > blocking on some socket operation (select, write, writev).  When the
> > socket becomes ready, we unwind to ap_bwrite's caller (ap_send_fd_length
> > for example) who iterates a loop to read the next piece of data from the
> > file/CGI pipe.  This is a back pressure mechanism that limits the amount
> > of data that Apache needs to buffer at any instant, which I think is a
> > Good Thing.  I'd hate to loose it with filters.
> The same back pressure should exist with either layer mechanism.

This is untrue.

The hook-based mechanism completely decouples the filtered-data generation
from the output. The filter is expected to return the *complete* filtered
output through the iovec.

In the link-based approach, we actively call "down" through the layers,
ending up at the network. If the network blocks, then the filter(s) will
pause in its content generation.


Greg Stein,

View raw message