hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arash Joorabchi <arash.joorab...@ul.ie>
Subject problem with multithreading and proxy
Date Mon, 09 Feb 2009 12:53:08 GMT

Hi all,

I am using MultiThreadedHttpConnectionManager in httpClient 3.1 to send 10
get methods to the Library of Congress SRU server
(http://z3950.loc.gov:7090/) concurrently each time. After running about 35
queries I get “server failed to respond” exceptions. In this setting I do
not set the httpClient  proxy parameters directly and instead use the
following lines in the applications’ main method 

System.getProperties().put( "proxySet",  config.get_proxySet() );
System.getProperties().put( "proxyHost", config.get_proxyHost() );
System.getProperties().put( "proxyPort", config.get_proxyPort() );

If I set the httpClient proxy parameters directly instead of “server failed
to respond” exception I get this exception: “HTTPclinet.fetch Method failed:
HTTP/1.1 502 Proxy Error (The specified network name is no longer available. 

When this occurs even the Internet Explorer fails to fetch the specified URI

however it still works with other URIs like Google.

To prevent this problem I have to send about 10 queries wait for 10 seconds
and then send another 10 queries.  Using this setting I ran about 20,000
queries overnight successfully.

The most puzzling is that if I do not change the URL and send the same query
over and over again no exception happens.

At this point I only can think of two things happening here:

1. My proxy server does not respond after a while because it thinks I am an
internal DOS attacker.

2. The server thinks I am a DOS attacker and does not respond if I send the
queries to fast.

I would appreciate your suggestions.


1. I do release the connections after each use.
2. I have tried sending queries with the connection close header.
3. I also tried another setting where I run 10 queries close/release
everything shutdown the connection manager and create a new one and send
another 10 queries.   

Arash Joorabchi

View this message in context: http://www.nabble.com/problem-with-multithreading-and-proxy-tp21912219p21912219.html
Sent from the HttpClient-User mailing list archive at Nabble.com.

To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

View raw message