hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kim B. Andersen" <KI...@tdc.dk>
Subject SV: Slow to open connection after an hour or so
Date Thu, 05 Jan 2006 07:38:11 GMT
Hi

thanks for responding


My ideas:
* Is there a chance that you are exhausting the connection pool by never 
returning your connections? New connection requests would then block 
until one gets available

Does simple also have a pool? - because I have tryed both with MultiThreadedHttpConnectionManager
and without and I still see the problem. What I do is something like this

httpclient client = new hhtpClient();
method = new getMethod(url);
client.execute(method);
...
...
method.releaseconnection();

How can I see if the pool is empty?

* Maybe attach a debugger / profiler or use jconsole
I have never played with these tools so I don't no how at the moment. I will try to look into
it, but if you can recommend some website to read or describe how to - I would appreciated
alot.

* Anything in the logs?
hope it's readable

2006/01/03 14:46:49:601 CET [DEBUG] DefaultHttpParams - Set parameter http.useragent = Jakarta
Commons-HttpClient/3.0-rc4
2006/01/03 14:46:49:601 CET [DEBUG] DefaultHttpParams - Set parameter http.protocol.version
= HTTP/1.1
2006/01/03 14:46:49:601 CET [DEBUG] DefaultHttpParams - Set parameter http.connection-manager.class
= class org.apache.commons.httpclient.SimpleHttpConnectionManager
2006/01/03 14:46:49:601 CET [DEBUG] DefaultHttpParams - Set parameter http.protocol.cookie-policy
= rfc2109
2006/01/03 14:46:49:601 CET [DEBUG] DefaultHttpParams - Set parameter http.protocol.element-charset
= US-ASCII
2006/01/03 14:46:49:601 CET [DEBUG] DefaultHttpParams - Set parameter http.protocol.content-charset
= ISO-8859-1
2006/01/03 14:46:49:601 CET [DEBUG] DefaultHttpParams - Set parameter http.method.retry-handler
= org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@1e845c2
2006/01/03 14:46:49:601 CET [DEBUG] DefaultHttpParams - Set parameter http.dateparser.patterns
= [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE, dd-MMM-yy HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE,
dd-MMM-yyyy HH:mm:ss z, EEE, dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy
HH:mm:ss z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy HH:mm:ss
z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z, EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy
HH:mm:ss z]
2006/01/03 14:46:49:616 CET [DEBUG] HttpClient - Java version: 1.5.0_06
2006/01/03 14:46:49:616 CET [DEBUG] HttpClient - Java vendor: Sun Microsystems Inc.
2006/01/03 14:46:49:616 CET [DEBUG] HttpClient - Java class path: T:\sun\lib\rt13.jar;C:\development\mrtg\classes;T:\sun\jdk\1.5.0_6\jre\lib\charsets.jar;T:\sun\jdk\1.5.0_6\jre\lib\deploy.jar;T:\sun\jdk\1.5.0_6\jre\lib\javaws.jar;T:\sun\jdk\1.5.0_6\jre\lib\jce.jar;T:\sun\jdk\1.5.0_6\jre\lib\jsse.jar;T:\sun\jdk\1.5.0_6\jre\lib\plugin.jar;T:\sun\jdk\1.5.0_6\jre\lib\rt.jar;T:\sun\jdk\1.5.0_6\jre\lib\ext\dnsns.jar;T:\sun\jdk\1.5.0_6\jre\lib\ext\localedata.jar;T:\sun\jdk\1.5.0_6\jre\lib\ext\sunjce_provider.jar;T:\sun\jdk\1.5.0_6\jre\lib\ext\sunpkcs11.jar;T:\sun\lib\log4.jar;T:\sun\lib\jrobin-1.4.0.jar;T:\sun\lib\snmp-1.3.jar;T:\sun\lib\weblogic61.jar;Q:\prod\releases\Krump.jar;Q:\prod\releases\KrumpThirdParty.jar;T:\sun\lib\xmlrpc-1.2-b1.jar;C:\development\prod;T:\lib\codec\1.3\commons-codec-1.3.jar;T:\lib\HttpClient\3.0
rc4\commons-httpclient-3.0-rc4.jar;T:\jetbrains\intellij\4.5\lib\idea_rt.jar
2006/01/03 14:46:49:616 CET [DEBUG] HttpClient - Operating system name: Windows 2000
2006/01/03 14:46:49:616 CET [DEBUG] HttpClient - Operating system architecture: x86
2006/01/03 14:46:49:616 CET [DEBUG] HttpClient - Operating system version: 5.0
2006/01/03 14:46:49:647 CET [DEBUG] HttpClient - SUN 1.5: SUN (DSA key/parameter generation;
DSA signing; SHA-1, MD5 digests; SecureRandom; X.509 certificates; JKS keystore; PKIX CertPathValidator;
PKIX CertPathBuilder; LDAP, Collection CertStores)
2006/01/03 14:46:49:647 CET [DEBUG] HttpClient - SunRsaSign 1.5: Sun RSA signature provider
2006/01/03 14:46:49:647 CET [DEBUG] HttpClient - SunJSSE 1.5: Sun JSSE provider(PKCS12, SunX509
key/trust factories, SSLv3, TLSv1)
2006/01/03 14:46:49:647 CET [DEBUG] HttpClient - SunJCE 1.5: SunJCE Provider (implements RSA,
DES, Triple DES, AES, Blowfish, ARCFOUR, RC2, PBE, Diffie-Hellman, HMAC)
2006/01/03 14:46:49:647 CET [DEBUG] HttpClient - SunJGSS 1.0: Sun (Kerberos v5)
2006/01/03 14:46:49:647 CET [DEBUG] HttpClient - SunSASL 1.5: Sun SASL provider(implements
client mechanisms for: DIGEST-MD5, GSSAPI, EXTERNAL, PLAIN, CRAM-MD5; server mechanisms for:
DIGEST-MD5, GSSAPI, CRAM-MD5)


First hour

2006/01/03 14:46:49:926 CET [DEBUG] HttpConnection - Open connection to fastnetselvbetjening.tdconline.dk:443
2006/01/03 14:46:50:038 CET [DEBUG] header - >> "GET /Krump/Alivetest.do?ws HTTP/1.1[\r][\n]"
2006/01/03 14:46:50:038 CET [DEBUG] HttpMethodBase - Adding Host request header
2006/01/03 14:46:50:054 CET [DEBUG] header - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
2006/01/03 14:46:50:069 CET [DEBUG] header - >> "Host: fastnetselvbetjening.tdconline.dk[\r][\n]"
2006/01/03 14:46:50:069 CET [DEBUG] header - >> "[\r][\n]"
2006/01/03 14:46:52:757 CET [DEBUG] header - << "HTTP/1.1 200 OK[\r][\n]"
2006/01/03 14:46:52:757 CET [DEBUG] header - << "Date: Tue, 03 Jan 2006 13:46:46 GMT[\r][\n]"
2006/01/03 14:46:52:757 CET [DEBUG] header - << "Server: Apache[\r][\n]"
2006/01/03 14:46:52:757 CET [DEBUG] header - << "Content-Type: text/html;charset=ISO-8859-1[\r][\n]"
2006/01/03 14:46:52:757 CET [DEBUG] header - << "Pragma: No-cache[\r][\n]"
2006/01/03 14:46:52:757 CET [DEBUG] header - << "Cache-Control: no-cache[\r][\n]"
2006/01/03 14:46:52:757 CET [DEBUG] header - << "Expires: Thu, 01 Jan 1970 00:00:00
GMT[\r][\n]"
2006/01/03 14:46:52:757 CET [DEBUG] header - << "Set-Cookie: JSESSIONID=8365C495E31360D1A57074D553751975;Path=/Krump[\r][\n]"
2006/01/03 14:46:52:757 CET [DEBUG] header - << "Transfer-Encoding: chunked[\r][\n]"
2006/01/03 14:46:52:788 CET [DEBUG] HttpMethodBase - Cookie accepted: "$Version=0; JSESSIONID=8365C495E31360D1A57074D553751975;
$Path=/Krump"
2006/01/03 14:46:52:788 CET [DEBUG] HttpConnection - Input data available
2006/01/03 14:46:52:804 CET [WARN] HttpMethodBase - Going to buffer response body of large
or unknown size. Using getResponseAsStream instead is recommended.
2006/01/03 14:46:52:804 CET [DEBUG] HttpMethodBase - Buffering response body
2006/01/03 14:46:52:804 CET [DEBUG] HttpMethodBase - Resorting to protocol version default
close connection policy
2006/01/03 14:46:52:804 CET [DEBUG] HttpMethodBase - Should NOT close connection, using HTTP/1.1
2006/01/03 14:46:52:804 CET [DEBUG] HttpConnection - Releasing connection back to connection
manager.
2006/01/03 14:46:52:804 CET [DEBUG] MultiThreadedHttpConnectionManager - Freeing connection,
hostConfig=HostConfiguration[host=https://fastnetselvbetjening.tdconline.dk]
2006/01/03 14:46:52:804 CET [DEBUG] IdleConnectionHandler - Adding connection at: 1136296012804
2006/01/03 14:46:52:804 CET [DEBUG] MultiThreadedHttpConnectionManager - Notifying no-one,
there are no waiting threads


After the first hour

2006/01/04 07:58:50:230 CET [DEBUG] HttpConnection - Open connection to fastnetselvbetjening.tdconline.dk:443
2006/01/04 07:58:59:230 CET [DEBUG] header - >> "GET /Krump/Alivetest/alivetester1.html
HTTP/1.1[\r][\n]"
2006/01/04 07:58:59:230 CET [DEBUG] HttpMethodBase - Adding Host request header
2006/01/04 07:58:59:230 CET [DEBUG] header - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
2006/01/04 07:58:59:230 CET [DEBUG] header - >> "Host: fastnetselvbetjening.tdconline.dk[\r][\n]"
2006/01/04 07:58:59:230 CET [DEBUG] header - >> "[\r][\n]"
2006/01/04 07:58:59:449 CET [DEBUG] header - << "HTTP/1.1 200 OK[\r][\n]"
2006/01/04 07:58:59:449 CET [DEBUG] header - << "Date: Wed, 04 Jan 2006 06:58:58 GMT[\r][\n]"
2006/01/04 07:58:59:449 CET [DEBUG] header - << "Server: Apache[\r][\n]"
2006/01/04 07:58:59:449 CET [DEBUG] header - << "Content-Type: text/html[\r][\n]"
2006/01/04 07:58:59:449 CET [DEBUG] header - << "Content-Length: 120[\r][\n]"
2006/01/04 07:58:59:449 CET [DEBUG] header - << "Last-Modified: Mon, 07 Oct 2002 14:00:15
GMT[\r][\n]"
2006/01/04 07:58:59:449 CET [DEBUG] header - << "ETag: "120-1033999215000"[\r][\n]"
2006/01/04 07:58:59:449 CET [DEBUG] HttpMethodBase - Buffering response body
2006/01/04 07:58:59:449 CET [DEBUG] HttpMethodBase - Resorting to protocol version default
close connection policy
2006/01/04 07:58:59:449 CET [DEBUG] HttpMethodBase - Should NOT close connection, using HTTP/1.1
2006/01/04 07:58:59:449 CET [DEBUG] HttpConnection - Releasing connection back to connection
manager.
2006/01/04 07:58:59:449 CET [DEBUG] MultiThreadedHttpConnectionManager - Freeing connection,
hostConfig=HostConfiguration[host=https://fastnetselvbetjening.tdconline.dk]
2006/01/04 07:58:59:449 CET [DEBUG] IdleConnectionHandler - Adding connection at: 1136357939449
2006/01/04 07:58:59:449 CET [DEBUG] MultiThreadedHttpConnectionManager - Notifying no-one,
there are no waiting threads

Kim Andersen
-----Oprindelig meddelelse-----
Fra: Ortwin Gl├╝ck [mailto:odi@odi.ch] 
Sendt: 4. januar 2006 17:19
Til: HttpClient Project
Emne: Re: Slow to open connection after an hour or so


My ideas:
* Anything in the logs?
* Is there a chance that you are exhausting the connection pool by never 
returning your connections? New connection requests would then block 
until one gets available
* Maybe attach a debugger / profiler or use jconsole

Hell, it would be nice if HttpClient had some JMX beans to provide 
information about pools etc. at runtime. I'll add that as a requirement 
for 4.0.

Odi

Kim B. Andersen wrote:
> Hi
> 
> I'm devolping a program which grabs webpage every 5min and measure the
> time it takes. I have succesful used httpclient to get the pages and it
> works fine:). The problem is opening of connection in httpclient get
> very slow after and hour or so.The first hour opening a connection takes
> 50ms at max and after an hour it takes 10 seconds. Opening of connection
> gets fast if I restarte the program. Any Ideas what the problem could
> be?
> 
> I have tried the following/uses:
> 
> jvm version: 	1.5.0_6/1.4.2_05
> httpclient:	3.0 rc4/ 3.0 rc4
> I have tried both with proxy and out
> I have tried both MultiThreadedHttpConnectionManager and simple
> 
> Hope you can help me
> 
> /Kim Andersen
> 

-- 
[web]  http://www.odi.ch/
[blog] http://www.odi.ch/weblog/
[pgp]  key 0x81CF3416
        finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org


Mime
View raw message