hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: Half closed connections issue
Date Mon, 16 Mar 2009 15:33:17 GMT
On Mon, 2009-03-16 at 14:27 +0000, Sam Crawford wrote:
> Oleg,
> 
> I've implemented the IdleConnectionEvictor you suggested, and whilst it
> seems to do the job (demonstrated by watching the TCP connections in
> Wireshark), I'm seeing some curious entries in the log:
> 
> 16-Mar-2009 14:25:24 org.apache.http.impl.conn.IdleConnectionHandler remove
> WARNING: Removing a connection that never existed!
> 
> This is being caused by the line: connMgr.closeIdleConnections(30,
> TimeUnit.SECONDS);
> 
> This suggests to me that I'm doing something wrong here, but I'm at a loss
> as to what. Any suggestions would be most appreciated.
> 

This is a bug in HttpClient 4.0b2, which has already been fixed in SVN
trunk. It is pretty harmless, though. You can either ignore the log
entries for the time being and wait for the next public official release
or upgrade to the latest code snapshot. 

Oleg

> Thanks,
> 
> Sam
> 
> 
> 2009/3/2 Sam Crawford <samcrawford@gmail.com>
> 
> > Perfect, that should do the job nicely.
> >
> > Many thanks,
> >
> > Sam
> >
> >
> > 2009/3/2 Oleg Kalnichevski <olegk@apache.org>
> >
> > On Sun, 2009-03-01 at 22:34 +0000, Sam Crawford wrote:
> >> > Hello,
> >> >
> >> > I'm having an issue with HttpClient communicating reliably with
> >> webservers
> >> > behind a load balancer. In summary, the first request goes through fine
> >> and
> >> > the load balancer closes the server>client connection 30 seconds later,
> >> > leaving the TCP connection half open. This same connection is then
> >> accessed
> >> > again much later as the client tries to close its half and a timeout
> >> occurs
> >> > (as the load balancer forgot about it ages ago). The full flow is as
> >> > follows:
> >> >
> >> > 1. Client makes a request to balancer1.com (for example). A new TCP
> >> > connection is established (fresh three way handshake).
> >> > 2. Client issues GET request for some content, and all goes well.
> >> > 3. 30 seconds later, the server (balancer1.com) sends a FIN/ACK, and
> >> the
> >> > client dutifully responds with an ACK. (note: at this point the
> >> > server>client connection is closed, but the client>server connection
is
> >> > still open)
> >> > 4. A long time later (e.g. 4000 seconds in my testing) the client is
> >> asked
> >> > to make another request to balancer1.com
> >> > 5. The client host sends a FIN/ACK on the same connection as was
> >> established
> >> > in step #1. No response is received (as the load balancer has timed out
> >> this
> >> > connection presumably), and it retries the FIN/ACK for 30 seconds.
> >> > 6. The client gives up and establishes a new TCP connection and requests
> >> the
> >> > content, and all is well.
> >> >
> >> > The issue with the above is the approx 30 second delay that occurs as
> >> the
> >> > client is attempting to close a connection that has long since been
> >> > forgotten about by the other party. I realise that load balancer
> >> dropping
> >> > the connection is causing the problem, but I'm sure there must be a way
> >> I
> >> > can set the HttpClient to actively send a follow-up FIN/ACK if one is
> >> > received from the other end? Ideally I still want to have persistent
> >> > HTTP/1.1 connections enabled to this server, as there will be busy peak
> >> > periods and very quiet periods.
> >> >
> >>
> >> Sam,
> >>
> >> This actually a limitation of the classic (blocking) I/O model in
> >> general. Blocking sockets cannot react to any I/O events unless they are
> >> read from or written to by a thread. Persistent connections are kept in
> >> the pool inactive (detached from execution thread), so they are not able
> >> to detect changes in the connection state and react to them
> >> appropriately.
> >>
> >> The only way around this problem is running a connection eviction thread
> >> that wakes up at a regular interval and drops expired / idle connections
> >> from the connection pool. See sample below:
> >>
> >>
> >> http://svn.apache.org/viewvc/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientEvictExpiredConnections.java?view=markup
> >>
> >> Hope this helps
> >>
> >> Oleg
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>
> >>
> >


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message