hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laura Werner <la...@bevocal.com>
Subject RE: About Bug 14005, HttpMultiClient reusing closed connection, a dvice needed
Date Thu, 05 Dec 2002 19:07:15 GMT
Aurelien Pernoud wrote:

> Does anyone have any idea on how to definitely solve this 
> issue, instead of "hacking" the code to repeat the request
> if there was a broken pipe ?

That's not really a hack.  RFC 2616, section 8.1.4 says that "A client,
server, or proxy MAY close the transport connection at any time".  So the
client has to watch for broken pipes, or however else a closed connection
manifests itself, and recover gracefully.  Catching the error and repeating
the request is part of that, even though it feels disgusting.

However, I think there are a couple of things that the client can do to
minimize the occurrance of broken connections.  One is to watch for the
Keep-Alive headers that some servers like Apache send back:

 HTTP/1.1 200 OK
 Connection: Keep-Alive
 Keep-Alive: timeout=15, max=74

I think this means that Apache is going to close the connection 15 seconds
after it was opened, if it's not still in use.  (Or after it's been idle for
15 seconds.)  I haven't found any documentation on this yet.  Either way, we
could use this header to add a field to HttpConnection that indicatates when
the connection will go stale.  If a stale connection was retrieved from a
HttpConnectionManager, it would just be closed (or re-opened?) rather than
used.  I'm thinking of submitting a patch on this one of these days, once I
finish working on our current release.

Another way of handing the server-side closes is suggested in 8.1.4:
"Clients and servers SHOULD both constantly watch for the other side of the
transport close, and respond to it as appropriate."  Right now, HttpClient
is letting the connections just sit there; it's not looking for any sort of
i/o or close events on them.  And it would be very hard to do that in Java
without having a separate thread for each connection, which would be a big
burden.  Maybe with the new model in JDK 1.4's java.nio package, one of
these years.

Laura Werner

> -----Original Message-----
> From: Aurelien Pernoud [mailto:apernoud@sopragroup.com]
> Sent: Thursday, December 05, 2002 5:34 AM
> To: 'Commons HttpClient Project'
> Subject: RE: About Bug 14005, HttpMultiClient reusing closed 
> connection,
> advice needed
> > De : Adrian Sutton [mailto:adrian.sutton@ephox.com]
> >
> > The problem is occurring because the server times out the
> > connection so the
> > problem is almost always a configuration setting somewhere.
> > It may however
> > not be a configuration in the server itself, the firewall can
> > be configured
> > to time out connections as well etc.  My first step in
> > debugging any problem
> > with HttpClient is to telnet into port 80 of the server
> > myself so I can see
> > exactly what's going on and get a lot more control.  Enabling
> > the http wire
> > logs are also good but I find the output of telnet is easier
> > to follow and
> > lets me change around what's sent a bit more easily.
> >
> > Basically though, it will happen on both servers if you leave
> > it long enough
> > - it's only a matter of time... (queue spooky music).
> >
> > Not much more detail I can shed on the situation I'm afraid.
> Hi Adrian,
> First thanks for your explanation and time. Here are some 
> tests I've made :
> UNIX tests : AIX 4.3.3 and AIX 5.1.0.
> WIN Tests : win 2000 pro/server only
> JDK : 1.3.1 (adapted to the platform)
> Webapp server : Tomcat 4.1.12 (adapted to the platform)
> I tried many configuration to see what was working, what wasn't.
> My app is using httpclient to get streams from other hosts, 
> and for the
> moment only local hosts (I never use proxy), and not secure.
> First when HttpClient is running under win2000, it *never* 
> goes on broken
> pipe, even after 2 days without making any request to another 
> host, may the
> distant host be a UNIX or a WIN, or localhost.
> When HttpClient is running under UNIX (both AIX systems above 
> have the same
> results), here's what happen :
> - When making request to localhost (loopback), it goes on 
> broken pipe only
> after like 10 sec of inactivity. Very weird, must be some 
> conf inside AIX,
> we have to look for this.
> - When making requests to a distant WIN system, it works fine 
> for hours, but
> crashes after two days of inactivity.
> - I haven't tested yet making request from Unix to distant 
> UNIX, but I'll
> let you know what happen once done.
> I've seen your solution for the moment : try-catch 
> IOException and retry,
> but in my mind this is only a hack, cause it nevers happens under WIN
> environment. I'm not a socket specialist at all, but I've 
> spent time with
> one at work, and he tolds me that they encountered broken 
> pipe in other
> projects when using different systems, because systems 
> sometime have their
> own way to close a socket. (not sure of my translation in 
> english, sorry)
> The socket is not properly closed, here is a state under AIX 
> after only
> seconds, and that crashes :
> Does anyone have any idea on how to definitely solve this 
> issue, instead of
> "hacking" the code to repeat the request if there was a broken pipe ?
> Thanks,
> Aurélien Pernoud.
> --
> To unsubscribe, e-mail:   
> <mailto:commons-httpclient-dev-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: 
> <mailto:commons-httpclient-dev-help@jakarta.apache.org>

View raw message