hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Wade <David.W...@optimation.co.nz>
Subject RE: getMethod.getResponseBody() leaks handles
Date Mon, 11 Aug 2003 21:13:45 GMT
Oleg,

I tried testing on my home x86 Linux box under IBM1.4.1 and Sun1.4.2 and
monitoring the processes file descriptor usage under /proc/'pid'/fd and the
same problem does not arise.     

Am not sure it is the disk buffering though as the reason I discovered the
problem was I was using the httpclient to monitor a process once a minute,
yet when that process was shutdown it showed some 45 sockets were connected.
As only the httpclient had been connecting to the process and only once a
minute, thus I had thought the issue was socket related not file handle
related.

Further...

The difference in behavior between the two routines seems to be caused by
line 689 in HttpMethodBase in method getResponseBody()

                    setResponseStream(null);

When I subsequently call  getMethod.releaseConnection() it has no effect as
the responseStream is null so ensureConnectionRelease() is never called
which means the httpconnectionmanager is never informed and thus (a leap)
the connection is not closed?

Thanks for following up on this.   I need to know whether the issue affects
some s/w I have just delivered to a customer.   They would not be happy with
me sending them a development release no matter how much I tested.

Regards
David

-----Original Message-----
From: Kalnichevski, Oleg [mailto:oleg.kalnichevski@bearingpoint.com] 
Sent: Monday, 11 August 2003 8:27 p.m.
To: Commons HttpClient Project
Subject: RE: getMethod.getResponseBody() leaks handles


David,

GetMethod contains an outdated disk buffering mechanism that has been long
deprecated but not removed for backward compatibility reason. The disk
buffering can well be causing the reported file handles leak. Curiously
enough, disk buffering is supposed to be off by default. Since you do not
explicitly turn disk buffering on, it should stay off.

Anyways, I'll try to reproduce the problem and get back to you as soon as I
know more. Meanwhile, if you happen to be in an adventurous mood, you may
try out the development version of HttpClient 2.1 with disk buffering
completely removed. Be warned, though, the development branch of HttpClient
is no longer completely 2.0 API compatible. Some minor tweaks to your code
may be necessary.

http://cvs.apache.org/builds/jakarta-commons/nightly/commons-httpclient/

Cheers

Oleg

-----Original Message-----
From: David Wade [mailto:David.Wade@optimation.co.nz]
Sent: Monday, August 11, 2003 07:01
To: 'commons-httpclient-dev@jakarta.apache.org'
Subject: getMethod.getResponseBody() leaks handles


HTTP-CLIENT 2.0 RC1
SUN JDK 1.4.2 
WIN-XP SP1 (leave me alone)

Have noticed that the use of getResponseBody() on GetMethod leaks file
handles as compared to getResponseBodyAsStream().   Bizzar but true.   The
following examples are for bug/feature demonstration only.   I tested
against a local Tomcat.  When repeating the tests change to an available
server/page.

Try running the following and monitoring the Java process with the task
manager with the additional column "handles" selected and run over a ten
minutes period.  Ater ten minutes the handle count is at 827 (on my
machine).

  // Leaking handles example
  public static void main(String[] args) {
    while (true) {
      HttpClient client = null;
      GetMethod method = null;

      try {
        client = new HttpClient();

        method = new GetMethod("http://localhost:8080");

        client.executeMethod(method);

        int httpStatus = method.getStatusCode();

        String responseBody = new String(method.getResponseBody());

        System.out.println("status: " + httpStatus);
      }
      catch (Throwable t) {}
      finally {
        try {
          method.releaseConnection();
        }
        catch (Throwable t) {}
      }

      try {
        Thread.sleep(100);  // gives time for garbage etc.
      }
      catch (InterruptedException ie) {}
    }
  }
  


In comparison, then try the following code using getResponseAsStream().
After ten minutes it uses just 339 handles.

  // No leaking handles example
  public static void main(String[] args) {
    while (true) {
      HttpClient client = null;
      GetMethod method = null;

      try {
        client = new HttpClient();

        method = new GetMethod("http://localhost:8080");

        client.executeMethod(method);

        int httpStatus = method.getStatusCode();

        InputStream is = method.getResponseBodyAsStream();
        
        byte[] buffer = new byte[16000];

        int len = is.read(buffer);

        byte[] body = new byte[len];

        System.arraycopy(buffer, 0, body, 0, len);

        String responseBody = new String(body);

        System.out.println("status: " + httpStatus);
      }
      catch (Throwable t) {}
      finally {
        try {
          method.releaseConnection();
        }
        catch (Throwable t) {}
      }

      try {
        Thread.sleep(100);  // gives time for garbage etc.
      }
      catch (InterruptedException ie) {}
    }
  }

This would 'appear' to be a serious bug.  Ten minutes and ~500 handles lost,
so what after a day...  

Have had a quick debug step thru of getResponseBody() and it does appear to
not execute any of clean up code, that happens in the second example.

Obviously I have a workaround, but feedback on whether other have the same
result (on Solaris would be interesting).

Ta. 
________________________________________________________________
David Wade              Senior Consultant
Optimation NZ Ltd       Phone :+64-9-3097918     
L4, Optimation House    DDI   :+64-9-3075561
43 College Hill Rd      Mobile:it doesn't work.
PO Box 106-104          mailto:david.wade@optimation.co.nz
Auckland, New Zealand   Web   :http://www.optimation.co.nz
 
 
NOTE: This electronic mail message together with any attachments 
is confidential. If you are not the intended  recipient, please 
e-mail us immediately and destroy this message. You may not copy, 
disclose or use the contents in any way. Thank you. 

---------------------------------------------------------------------
To unsubscribe, e-mail:
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail:
commons-httpclient-dev-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail:
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail:
commons-httpclient-dev-help@jakarta.apache.org

Mime
View raw message