apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "William A. Rowe, Jr." <wr...@rowe-clan.net>
Subject Re: Win32 apr_sockets; Random notes from the 'oh, duh' department
Date Thu, 24 Apr 2003 00:22:56 GMT
At 05:24 PM 4/23/2003, Bill Stoddard wrote:
>William A. Rowe, Jr. wrote;
>>  Serious Problem #1: Blocking Socket, apr_sendv() invokes WSASend()
>>with NULL completion arguments... according to WinSock2 this should
>>be a sync, blocking send.  Put a huge (4MB), mmap'ed region into that iovec, and it
will be sent - BUT IT IS SENT ASYNC!  So later we close the
>>socket and the transmission is aborted.

>Humm...  I'll ruminate on this some... there may be another solution. You sure the WSASend
call is not returning a timeout expiration?

No, the strangest bit is that I'm running on W2K SP3... but this sounds
very much like the XP afd.sys bug.  We begin the transfer and the function 
returns immediately ... it does continue to send the entire 4MB.  
Upon socket close, Win32 blocks for the file to complete.  But it doesn't 
seem to do so predictably.  No matter what, the buffer could move out
from under WSASend, so it's a big bug.  But the truncation could also
have to do with my need to do a linger close.

/me wonders if we linger correctly when doing a TransmitFile and 
attempting to reuse the socket.

>>  Solution?  Apparently we just can't trust sync send/recv, and I guess
>>it is time to simply surrender and make them all async with a completion
>>event, irrespective of timeouts.  Then wait on that event INFINITE or by the so_timeout
>>  Question; do we simply use the socket itself as the event handle?  
>Should be fine as long as multiple threads never attempt to touch the socket at the same
time. apr_sendfile uses the socket as the event handle and AFAIK it has never caused a problem.

Right.  Once an MPM passes a socket to a given thread, it must be

>>Or do we create one event for each socket that we can later use for all
>>sorts of goodness such as a WSAAsyncSelect()-style poll?  Two threads
>>won't be looking at the same socket, so I think an event handle in each
>>apr_socket_t would be useful.
>I'd prefer to not use a seperate event handle (for performance reasons) unless we have
a clear demonstrated fully thought out need to. MHO.

I'm also rewriting poll using WSAEventSelect() (not AsyncSelect - that's
the one that posts messages to hWnds.)  I will toss in some ifdefs that
let us play with both models (socket-as-event v.s. discrete event handle.)

>>  Serious Problem #2: Win32 doesn't support socket timeouts.  We are
>>using raw WSASend/WSARecv, blocking.  This just isn't good.
>Sure it does, just not the same way as most unix systems. Unix uses nonblocking i/o followed
by a select to do timeout. 

Huh?  I'm describing what happens if we timeout within a single operation.
Not leading 'up to' that operation.

>This is horribly inefficient on Windows where sys call overhead is high.  
>Windows uses the builtin setsockopt(SO_RCVTIMEOUT) timeout 
>mechanisms (which are part of the BSD socket spec but is not 
>implemented on Unix).  If, as you are reporting, WSA* calls are not 
>really working as they should, then we should probably just do them 
>async and wait for the io completion.

Ok... according to every document on WinSock2 I've read, it appears that
SO_RCVTIMEOUT isn't supported on Win32.  I could be misreading, or
just out of my mind, but it appears clearly documented - as unsupported.

>>  Solution?  Same as for #1 above.

Of course if we solve the problems above, this one comes free even if
SO_RCVTIMEOUT isn't implemented.

>>  Serious Problem #3: Using WaitForSingleObject() with our 'socket'
>>timeout value is rather bogus - it would measure the total elapsed time,
>>not the interval between packet transmissions.
>>  Solution?  Well, if WinSock2 had some sort of 'progress' indication, bytes
>>sent or received so far for a completion-based request - then we could just
>>sample and assure ourselves that something had happened.  As it is we
>>just can't do that.  Effectively, it looks like any huge send over a slow wire
>>will be timed out based on any sane setting for a timeout value.
>That should not be the case. Last time I checked, I though we set the timeout 
>then sent 64K (?) chunks then reset the timeout for the next 64k chunk. 
>This is -eaxactly- what 1.3 does on Unix (where signals are used as the 
>timeout mechanism). This has someone changed that w/o my noticing? 
>I'll check later...

No, especially not in the case of mmaps.  The way filters, brigades and
buckets work, you can end up with a 16MB sendfile or 4MB mmap.  In
the later case, you may have 4MB * 16 iovecs in a single WSASend.

View raw message