httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yann Ylavic <ylavic....@gmail.com>
Subject Re: mod_proxy, oooled backend connections and the keep-alive race condition
Date Sun, 08 Dec 2013 23:30:34 GMT
On Sun, Dec 8, 2013 at 11:51 AM, Micha Lenk <micha@lenk.info> wrote:

>
> Am 05.12.2013 22:03, schrieb Yann Ylavic:
> > When mod_proxy encounters a closed (reused) connection while forwarding
> > the client's request, it has just sent the request once, right?
>
> That depends on the actual state. What Thomas is talking about (I know
> because I worked with him on the same product he is referring to here)
> is the state where you have everything in place to send out the next
> request headers on a connection that was established by a previous
> request and that was kept open using HTTP Keep-Alive. Let me stress the
> fact that in this state you haven't sent a single bit of the current
> request over the wire yet.
>

Yes, here we are.



>
> Now, if trying to send the first bytes of the request immediately fails
> because the socket isn't connected anymore (e.g. EPIPE), you *know* that
> exactly *none* bits of your current request reached the server. For this
> reason it should be safe to retry by establishing a new connection to
> the server and to try to send the request again over the newly
> 
> established connection.


The send() will never fail at this point if the peer FIN-closed the
connection (EPIPE or whatever on non-unixes), since it is only half-closed.
You can't read anymore, but you still can write, and this is precisely why
the error log is "error *reading* status line" and not "error writing ...".
A second send() could fail however, though possibly with ECONNRESET, and
you are no better here.

The point is that mod_proxy already checks whether the connection is alive
before trying to send a (successive) request on the kept alive connection,
by using a recv(MSG_PEEK) (see ap_proxy_is_socket_connected).
If that fails, a new connection is established and the request is sent on
it, this is already the current behaviour.
The race condition is when this check succeeds, but the peer closes the
connection before the request reaches it (in the meantime).


>
> In the sense of getting the reverse proxy request correctly processed by
> the backend server, this approach *does* fix the keep-alive race
> condition issue where the server closes the backend connection because
> it sees no (new) client request for too long time -- IIRC this is what
> this discussion started with.
>

It does *not* fix the race, nothing can, and the proxy can't know whether
its request as been processed by the peer based on the (second) send()
error.

IMHO, the best that can be done is HTTP ping for POST requests (using
"Expect: 100-continue", already implemented), retry idempotent requests
(TODO), reduce the race time (my proposal/patch), and let client client
decide for others ("proxy-initial-not-pooled" can help it).
 

>
> I hope I could shed some more some light on this issue and on the idea
> how to possibly tackle it in Apache httpd.
>

Thanks, I really wish it could be, and Thomas prove me wrong!

Regards,
Yann.

Mime
View raw message