Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@httpd.apache.org
Received-SPF: softfail (asf.osuosl.org: transitioning domain of
 psusi@cfl.rr.com does not designate 209.208.122.204 as permitted sender)
Message-ID: <4381F407.6090606@cfl.rr.com>
Date: Mon, 21 Nov 2005 11:21:27 -0500
From: Phillip Susi <psusi@cfl.rr.com>
User-Agent: Thunderbird 1.5 (Windows/20051025)
MIME-Version: 1.0
To: dev@httpd.apache.org
Subject: Re: Deleting the async-dev branch
References: <E7D02657-A290-4B02-9CB9-DF8877E31C2F@apache.org>
 <200511201046.16378.nick@webthing.com> <43805F3D.2070709@ukf.net>
 <4380EBE3.5030005@cfl.rr.com>
 <D0BF3D9D-0E1B-4A84-A83A-DE414376AFD1@apache.org>
In-Reply-To: <D0BF3D9D-0E1B-4A84-A83A-DE414376AFD1@apache.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Having a thread block to page in data that isn't currently in the cache 
is fine, as long as other threads are able to handle all the other 
requests.  What I would really like to see is support for zero copy 
sends without using sendfile().  Right now I believe that apache is 
calling sendfile() on small chunks of data at a time, so that it can 
make sure the transfer is not stalled.  This tends to let the pipeline 
stall for short periods between each sendfile() call.  Also sendfile() 
can not be used at all to send data that is not in a file on disk.  For 
instance, the output of a cgi script, or data that has been encrypted to 
send over SSL.

On NT you can direct the kernel to set the send buffer size to zero ( 
setsockopt I think it was ) and that will prevent it from making the 
user->kernel copy when you send.  To keep the pipeline full you just 
have to overlap multiple async send requests.  This allows you to get 
sendfile() performance without its drawbacks.  I hope to one day see 
linux and apache support this.

I have wondered though, what it would be like to be able to 
asynchronously page fault in the cache pages to send, rather than 
relying on the send() call to synchronously fault in the pages if 
needed.  That would save threads from blocking on disk IO, which might 
be nice.

Brian Pane wrote:
> The async write completion stuff that's now in the trunk does nonblocking
> network writes, and it uses sendfile to do zero-copy transfer from the 
> filesystem
> cache to the network on platforms that support it, but it doesn't try to 
> do file
> I/O asynchronously.  Thus it's possible for the current code to block on a
> disk read while trying to do a nonblocking socket write.  (The Event MPM
> compensates for this by allowing multiple threads to do sendfile calls on
> different sockets concurrently; if one thread happens to block on a
> physical disk read, it doesn't delay other connections.)  Having a means
> of doing nonblocking filesystem I/O, via either aio or something like
> mincore checks, might be a nice addition.
> 
> Brian
> 
>