hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Wade <David.W...@optimation.co.nz>
Subject RE: getMethod.getResponseBody() leaks handles
Date Mon, 11 Aug 2003 22:10:22 GMT
Mike,

Thanks for the confirmation.

However, I can see no justification for the difference in behavior between
getResponseBody and getResponseBodyAsStream  -especially as all
getResponseBody does is call getResponseBodyAsStream.   The difference is
caused solely by the line 689 mentioned below and it should be removed.

Yes I could change to a single instance of the http client, but I really
don't see why I have to keep an open connection between get method
invocations.  If I have finished with the connection and have asked for it
to be released, then that is what should happen.

David

-----Original Message-----
From: Michael Becke [mailto:becke@u.washington.edu] 
Sent: Tuesday, 12 August 2003 9:32 a.m.
To: Commons HttpClient Project
Subject: Re: getMethod.getResponseBody() leaks handles


Hi David,

Yes, the sample code you posted leaks HttpConnections and therefore  
Sockets.  This is because every time an instance of HttpClient is  
created it also creates an instance of SimpleHttpConnectionManager.   
This connection manager creates a new connection the first time it is  
used.  This means one connection is lost each time through the loop.

Calling getResponseBody() implicitly releases the connection, though  
this does not necessary mean the connection gets closed.  Calling  
releaseConnection() in the case of a successful getResponseBody() is  
redundant and therefore has no effect.  Having said that  
releaseConnection() should still always be called to ensure that a  
connection is released in the exception case.

The discrepancy between the two cases is most likely due to OS policies  
for handling unused connections.  It looks like Linux eventually closes  
the lost sockets, but Windows does not.

I would suggest reusing a single instance of a HttpConnectionManager.   
Either a SimpleHttpConnectionManager or a  
MultiThreadedHttpConnectionManager depending on your circumstances.

Mike

On Monday, August 11, 2003, at 05:13 PM, David Wade wrote:

> Oleg,
>
> I tried testing on my home x86 Linux box under IBM1.4.1 and Sun1.4.2
> and
> monitoring the processes file descriptor usage under /proc/'pid'/fd  
> and the
> same problem does not arise.
>
> Am not sure it is the disk buffering though as the reason I discovered
> the
> problem was I was using the httpclient to monitor a process once a  
> minute,
> yet when that process was shutdown it showed some 45 sockets were  
> connected.
> As only the httpclient had been connecting to the process and only  
> once a
> minute, thus I had thought the issue was socket related not file handle
> related.
>
> Further...
>
> The difference in behavior between the two routines seems to be caused
> by
> line 689 in HttpMethodBase in method getResponseBody()
>
>                     setResponseStream(null);
>
> When I subsequently call  getMethod.releaseConnection() it has no
> effect as
> the responseStream is null so ensureConnectionRelease() is never called
> which means the httpconnectionmanager is never informed and thus (a  
> leap)
> the connection is not closed?
>
> Thanks for following up on this.   I need to know whether the issue  
> affects
> some s/w I have just delivered to a customer.   They would not be  
> happy with
> me sending them a development release no matter how much I tested.
>
> Regards
> David
>
> -----Original Message-----
> From: Kalnichevski, Oleg [mailto:oleg.kalnichevski@bearingpoint.com]
> Sent: Monday, 11 August 2003 8:27 p.m.
> To: Commons HttpClient Project
> Subject: RE: getMethod.getResponseBody() leaks handles
>
>
> David,
>
> GetMethod contains an outdated disk buffering mechanism that has been
> long
> deprecated but not removed for backward compatibility reason. The disk
> buffering can well be causing the reported file handles leak. Curiously
> enough, disk buffering is supposed to be off by default. Since you do  
> not
> explicitly turn disk buffering on, it should stay off.
>
> Anyways, I'll try to reproduce the problem and get back to you as soon
> as I
> know more. Meanwhile, if you happen to be in an adventurous mood, you  
> may
> try out the development version of HttpClient 2.1 with disk buffering
> completely removed. Be warned, though, the development branch of  
> HttpClient
> is no longer completely 2.0 API compatible. Some minor tweaks to your  
> code
> may be necessary.
>
> http://cvs.apache.org/builds/jakarta-commons/nightly/commons-
> httpclient/
>
> Cheers
>
> Oleg
>
> -----Original Message-----
> From: David Wade [mailto:David.Wade@optimation.co.nz]
> Sent: Monday, August 11, 2003 07:01
> To: 'commons-httpclient-dev@jakarta.apache.org'
> Subject: getMethod.getResponseBody() leaks handles
>
>
> HTTP-CLIENT 2.0 RC1
> SUN JDK 1.4.2
> WIN-XP SP1 (leave me alone)
>
> Have noticed that the use of getResponseBody() on GetMethod leaks file
> handles as compared to getResponseBodyAsStream().   Bizzar but true.    
> The
> following examples are for bug/feature demonstration only.   I tested
> against a local Tomcat.  When repeating the tests change to an
> available
> server/page.
>
> Try running the following and monitoring the Java process with the 
> task manager with the additional column "handles" selected and run over a
> ten
> minutes period.  Ater ten minutes the handle count is at 827 (on my
> machine).
>
>   // Leaking handles example
>   public static void main(String[] args) {
>     while (true) {
>       HttpClient client = null;
>       GetMethod method = null;
>
>       try {
>         client = new HttpClient();
>
>         method = new GetMethod("http://localhost:8080");
>
>         client.executeMethod(method);
>
>         int httpStatus = method.getStatusCode();
>
>         String responseBody = new String(method.getResponseBody());
>
>         System.out.println("status: " + httpStatus);
>       }
>       catch (Throwable t) {}
>       finally {
>         try {
>           method.releaseConnection();
>         }
>         catch (Throwable t) {}
>       }
>
>       try {
>         Thread.sleep(100);  // gives time for garbage etc.
>       }
>       catch (InterruptedException ie) {}
>     }
>   }
>
>
>
> In comparison, then try the following code using 
> getResponseAsStream(). After ten minutes it uses just 339 handles.
>
>   // No leaking handles example
>   public static void main(String[] args) {
>     while (true) {
>       HttpClient client = null;
>       GetMethod method = null;
>
>       try {
>         client = new HttpClient();
>
>         method = new GetMethod("http://localhost:8080");
>
>         client.executeMethod(method);
>
>         int httpStatus = method.getStatusCode();
>
>         InputStream is = method.getResponseBodyAsStream();
>
>         byte[] buffer = new byte[16000];
>
>         int len = is.read(buffer);
>
>         byte[] body = new byte[len];
>
>         System.arraycopy(buffer, 0, body, 0, len);
>
>         String responseBody = new String(body);
>
>         System.out.println("status: " + httpStatus);
>       }
>       catch (Throwable t) {}
>       finally {
>         try {
>           method.releaseConnection();
>         }
>         catch (Throwable t) {}
>       }
>
>       try {
>         Thread.sleep(100);  // gives time for garbage etc.
>       }
>       catch (InterruptedException ie) {}
>     }
>   }
>
> This would 'appear' to be a serious bug.  Ten minutes and ~500 handles
> lost,
> so what after a day...
>
> Have had a quick debug step thru of getResponseBody() and it does
> appear to
> not execute any of clean up code, that happens in the second example.
>
> Obviously I have a workaround, but feedback on whether other have the
> same
> result (on Solaris would be interesting).
>
> Ta. ________________________________________________________________
> David Wade              Senior Consultant
> Optimation NZ Ltd       Phone :+64-9-3097918
> L4, Optimation House    DDI   :+64-9-3075561
> 43 College Hill Rd      Mobile:it doesn't work.
> PO Box 106-104          mailto:david.wade@optimation.co.nz
> Auckland, New Zealand   Web   :http://www.optimation.co.nz
>
>
> NOTE: This electronic mail message together with any attachments
> is confidential. If you are not the intended  recipient, please
> e-mail us immediately and destroy this message. You may not copy,
> disclose or use the contents in any way. Thank you.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:  
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:  
> commons-httpclient-dev-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail:
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail:
commons-httpclient-dev-help@jakarta.apache.org

Mime
View raw message