hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Becke <be...@u.washington.edu>
Subject Re: HttpClient under load
Date Wed, 02 Feb 2005 13:38:40 GMT
Hi Prasad,

Given the scenario you describe below you are more likely leaking one  
connection/socket per request.  You definitely want to reuse instances  
of HttpClient  and especially instances of a ConnectionManager.  You  
can either use the multi-threaded connection manager from all threads  
or have a dedicated SimpleHttpConnectionManager per thread.  Please  
have a look at the performance guide for some more ideas  
<http://jakarta.apache.org/commons/httpclient/3.0/performance.html>.

Also please note that calling manager.closeIdleConnections(1000L) only  
closes connections that have already been idle for 1000ms.  It does not  
continuously poll for idle connections, closing them when they've been  
idle for 1000ms.

Mike

On Feb 1, 2005, at 11:14 PM, Prasad Sethumadhavan wrote:

> Hi,
> I need some help or pointers regarding using the HttpClient under heavy
> load. I am currently using the HttpClient under moderate load (50
> concurrent threads). Each thread instatiates a new HttpClient client,
> uses it and does the method.releaseConnection() before exiting.
>
>         SimpleHttpConnectionManager manager = new
> SimpleHttpConnectionManager();
>         manager.getParams().setStaleCheckingEnabled(true);
>         manager.closeIdleConnections(1000L);
>         manager.getParams().setConnectionTimeout(
>                 (int) DEFAULT_CONNECTION_TIMEOUT);
>
>         httpclient = new HttpClient(manager);
>         httpclient.getParams().setCookiePolicy(
>                 CookiePolicy.BROWSER_COMPATIBILITY);
>
> However when I run this setup under load, I keep getting socket
> exceptions on the client side. I am using tomcat web server and I have
> increased the max-processor & access-count. I don't see any errors or
> exception in the tomcat log.
>
> However I keep getting a lot of these client socket bind exceptions
> after sometime. I guess it is happening due to the unavailability of
> ports. I would like to know some best practices in using the HttpClient
> under load. I am sure tha it is my bad, but wanted to get some input
> from the user/dev community. Should I have a pool of HttpClient objects
> and reuse them? Should I use the multi-threaded connection manager? I
> need to run this under heavier load and need to fix this soon. Any  
> help,
> code snippets, or guidance would be greatly appreciated. Please let me
> know if you need any other information.
>
> Thanks in advance
> Prasad
>
>
> Client StackTrace:
>
> [ERROR] 01 Feb 2005 20:43:56,177 NDC[] T[Thread-20] HTTP          HTTP
> I/O transport exception occurred
> java.net.BindException: Address already in use: connect
> 	at java.net.PlainSocketImpl.socketConnect(Native Method)
> 	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
> 	at
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
> 	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
> 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:364)
> 	at java.net.Socket.connect(Socket.java:507)
> 	at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> 	at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso 
> r
> Impl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:585)
> 	at
> org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSo 
> c
> ket(ReflectionSocketFactory.java:128)
> 	at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.cre 
> a
> teSocket(DefaultProtocolSocketFactory.java:124)
> 	at
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java: 
> 70
> 5)
> 	at
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpM 
> e
> thodDirector.java:359)
> 	at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMeth 
> o
> dDirector.java:168)
> 	at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java: 
> 3
> 93)
> 	at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java: 
> 3
> 24)
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:  
> httpclient-user-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message