hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Crawford <samcrawf...@gmail.com>
Subject Re: ThreadSafeClientConnectionManager and KeepAlive connnections
Date Thu, 08 Sep 2011 14:01:49 GMT
Thomas,

SO_REUSEADDRESS is not really relevent to HttpClient's connection pooling
mechanism; it's a hint to the OS's TCP stack to reuse an existing TCP socket
to the same target in the TIME_WAIT state.

I cannot see anything obviously wrong with what you are doing. Sanity check:
What parameters are you using for the TSCCM? i.e. What have you passed to
TSCCM.setMaxTotal(), setDefaultMaxPerRoute(), etc, or are you defining
seperately on each HttpRoute?

One potential cause:
The third line of your log says "Getting free connection
[HttpRoute[{s}->https://xxxx.intranet.com:410]][null]", where the 'null' is
'state'. When you come to pool the connection on the final line, it looks
like the state has been set to some DN (perhaps a client/server SSL
certificate?). I'm not sure what this state variable indicates, but I
suspect the fact that you're requesting the connection with a null state and
pooling it with a non-null state may be the cause of your problem.

Thanks,

Sam



On 8 September 2011 14:21, Thomas Kunnumpurath
<thomas.kunnumpurath@db.com>wrote:

> Classification: *Public *
>
> Hi,
>
> I am using the ThreadSafeClientConnectionManager in the HttpClient 4.1 API
> to submit secure requests to a webserver (nginx). From the debug logs, I do
> see that the connections are being pooled and reused, however, if I use
> netstat or tcpview I see that new TCP connections are constantly being
> created in the ESTABLISHED state and the old ones are put in TIME_WAIT state
> with no further activity. What I would expect to see is that the same TCP
> handle would be reused for a period of time. Can someone explain to me why a
> KeepAlive connection kicked off by HttpClient would not be reused? I can
> confirm that the web server supports keep alive and that the server is
> reporting that my program is closing the keep-alive connections****
>
> ** **
>
> Please see a sample of my DEBUG logs below:****
>
> ** **
>
> DEBUG 03:07:03.490 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.t.ConnPoolByRoute - [HttpRoute[{s https://xxxx.intranet.com:410]]
> total kept alive: 10, total issued: 0, total allocated: 10 out of 10****
>
> DEBUG 03:07:03.490 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.DefaultClientConnection - Connection closed****
>
> DEBUG 03:07:03.490 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.t.ConnPoolByRoute - Getting free connection [HttpRoute[{s}->
> https://xxxx.intranet.com:410]][null]****
>
> DEBUG 03:07:03.490 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.DefaultClientConnectionOperator - Connecting to ://
> xxxx.intranet.com:410
> DEBUG 03:07:03.549 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.c.p.RequestAddCookies - CookieSpec selected: best-match****
>
> DEBUG 03:07:03.549 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.c.p.RequestAuthCache - Auth cache not set in the context****
>
> DEBUG 03:07:03.549 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.DefaultHttpClient - Attempt 1 to execute request****
>
> DEBUG 03:07:03.549 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.DefaultClientConnection - Sending request: PUT
> /bla/put?name=TG_AUD_3M  HTTP/1.1****
>
> DEBUG 03:07:03.552 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.DefaultClientConnection - Receiving response: HTTP/1.1 201 Created
> ****
>
> DEBUG 03:07:03.552 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.DefaultHttpClient - Connection can be kept alive for 60000
> MILLISECONDS****
>
> DEBUG 03:07:03.552 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.t.ThreadSafeClientConnManager - Released connection is reusable.
> ****
>
> DEBUG 03:07:03.552 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.t.ConnPoolByRoute - Releasing connection [HttpRoute[{s}->https://xxxx.intranet.com:410
> ]][CN=xxxxx.uk.db.com:Tesla, O=xxxx, DC=xx, DC=com]****
>
> DEBUG 03:07:03.552 - 08/09/2011 [pool-1-thread-43] []
> o.a.h.i.c.t.ConnPoolByRoute - Pooling connection [[HttpRoute[{s}->https://xxxx.intranet.com:410
> ]][CN=xxxxx.uk.db.com:Tesla, O=xxxx, DC=xx, DC=com];keep alive for 60000
> MILLISECONDS
>
> I have a PUT request twice every second and I see that on average 2 TCP
> connections are established on the server every second, with the previous 2
> being put in TIME_WAIT state.
>
> The Response received from the server is as follows:
>
> Response headers:****
>
> HTTP/1.1 201 Created****
>
> Server: nginx/1.0.4****
>
> Date: Thu, 08 Sep 2011 05:00:52 GMT****
>
> Content-Type: application/octet-stream****
>
> Content-Length: 8****
>
> Connection: keep-alive****
>
> ** **
>
> ** **
>
> HTTPSampleResult fields:****
>
> ContentType: application/octet-stream****
>
> DataEncoding: null****
>
> ** **
>
> Wire Logs below:****
>
> DEBUG 09:17:18.034 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - >>
> "PUT /tesla/put?name=TG_AUD_3M HTTP/1.1[\r][\n]"****
>
> DEBUG 09:17:18.034 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - >>
> "Content-Length: 3160[\r][\n]"****
>
> DEBUG 09:17:18.034 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - >>
> "Host: xxxxx:410[\r][\n]"****
>
> DEBUG 09:17:18.034 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - >>
> "Connection: Keep-Alive[\r][\n]"****
>
> DEBUG 09:17:18.034 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - >>
> "User-Agent: Apache-HttpClient/4.1.2 (java 1.5)[\r][\n]"****
>
> DEBUG 09:17:18.034 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - >>
> "[\r][\n]"****
>
> ……….****
>
> DEBUG 09:17:18.034 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - >>
> "[\n]"****
>
> DEBUG 09:17:18.112 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - <<
> "HTTP/1.1 201 Created[\r][\n]"****
>
> DEBUG 09:17:18.112 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - <<
> "Server: nginx/1.0.4[\r][\n]"****
>
> DEBUG 09:17:18.112 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - <<
> "Date: Thu, 08 Sep 2011 13:17:17 GMT[\r][\n]"****
>
> DEBUG 09:17:18.112 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - <<
> "Content-Type: application/octet-stream[\r][\n]"****
>
> DEBUG 09:17:18.112 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - <<
> "Content-Length: 8[\r][\n]"****
>
> DEBUG 09:17:18.112 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - <<
> "Connection: keep-alive[\r][\n]"****
>
> DEBUG 09:17:18.112 - 08/09/2011 [pool-1-thread-1] [] o.a.http.wire - <<
> "[\r][\n]"****
>
> ** **
>
> Shouldn’t the default behavior be to reuse the existing sockets (I’ve even
> set the SO_REUSEADDRESS to true, but to no avail). Assistance in
> troubleshooting this issue would be greatly appreciated.****
>
> ** **
>
> ** **
>
> Thanks,****
>
> ____________________________________________________
>
> [image: https://brandportal.intranet.db.com/img/modules/logo.gif]
>
> Thomas Kunnumpurath
> Assistant Vice President | Application Developer - AS Pricing
>
> Deutsche Bank AG, Filiale New York
> Global Rates IT
> 60 Wall Street, 10005-2836 New York, NY, USA
> Tel. +1(212)250-2269
> Email thomas.kunnumpurath@db.com****
>
>
> [image: https://brandportal.intranet.db.com/img/modules/claim.gif]****
>
> ** **
>
> ---
>
> This communication may contain confidential and/or privileged information.
> If you are not the intended recipient (or have received this communication
> in error) please notify the sender immediately and destroy this
> communication. Any unauthorized copying, disclosure or distribution of the
> material in this communication is strictly forbidden.
>
> Deutsche Bank does not render legal or tax advice, and the information
> contained in this communication should not be regarded as such.
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message