hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Speirs <wspe...@apache.org>
Subject Re: httpclient.defaultMaxPerRoute and maxTotalConnections questions
Date Tue, 06 Dec 2011 01:33:11 GMT
On Mon, Dec 5, 2011 at 6:07 PM, mpindyala <mohan.pindyala@rovicorp.com> wrote:
> We have a java web application which makes calls to 2 of our third party
> Services using httpclient.
> Following are the settings for
> org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManagerr. the numbers
> are ridiculously high
> httpclient.defaultMaxPerRoute = 10000
> httpclient.maxTotalConnections = 20000
> Socket and connection timeout on org.apache.http.params.HttpConnectionParams
> are set as
> httpclient.socketTimeout = 120000ms
> httpclient.connectionTimeout = 20000ms
> we have disabled the
> org.apache.http.params.HttpConnectionParams#setStaleCheckingEnabled = false.
> we have a cron process running which checks for idle connections using
> ClientConnectionManager object every 10 seconds and closes expired & idle
> connections.

What do you mean a "cron process"? A thread that checks? Why would you
do this at all? Why wouldn't you let the connection manager handle
this for you?

> We also have retry handler set on
> org.apache.http.impl.client.DefaultHttpClient. It retries once with a 40ms
> delay.
> So questions are as follows
> 1. Does above setting look ok for you ? if not what are the optimal numbers.
> Lets say there are 4 third party services we have to call , how should we
> set these numbers to support that.

There are no real "optimal numbers" it is more a matter of the service
level agreement of the 3rd party and the expectations of your program.
You have a socket timeout of 2 minutes and a connection timeout of 20
seconds. Those seem high to me, but again, you might be doing
something which necessitates this.

> 2. We sometimes get following errors, which are not able to figure out if it
> is the third party server or httpclient issue. These errors show up during
> peak volume hitting our servers.
> a. java.net.SocketException: Connection reset

Someone correct me if I'm wrong, but I believe this means the server
close the connection on you. This brings up another important
question, are you making HTTP 1.0 or 1.1 requests? Keep-Alive?

> b. :java.net.SocketTimeoutException: Read timed out

This is somewhat straightforward, you tried to read, but didn't get data in 20s.

> c. org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for
> connection

This is also somewhat straightforward, there was, "A timeout while
waiting for an available connection from a connection manager." (from
the Javadocs) How many outstanding requests did you have? Could all
1000 be dealing with some long request?

I would also see if you can enable logging (this can kill performance)
and maybe scale-back your numbers a bit and see if you can get it to
happen again? My guess is that you're simply exhausting resources (#
connections, etc). How many requests were you making at peak times?

Hope this helps...


To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

View raw message