Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpClient User Discussion" <httpclient-users@hc.apache.org>
Received-SPF: pass (athena.apache.org: domain of lists@nabble.com designates
 216.139.236.158 as permitted sender)
Message-ID: <21912219.post@talk.nabble.com>
Date: Mon, 9 Feb 2009 04:53:08 -0800 (PST)
From: Arash Joorabchi <arash.joorabchi@ul.ie>
To: httpclient-users@hc.apache.org
Subject: problem with multithreading and proxy
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi all,

I am using MultiThreadedHttpConnectionManager in httpClient 3.1 to send 10
get methods to the Library of Congress SRU server
(http://z3950.loc.gov:7090/) concurrently each time. After running about 35
queries I get =E2=80=9Cserver failed to respond=E2=80=9D exceptions. In thi=
s setting I do
not set the httpClient  proxy parameters directly and instead use the
following lines in the applications=E2=80=99 main method=20

System.getProperties().put( "proxySet",  config.get_proxySet() );
System.getProperties().put( "proxyHost", config.get_proxyHost() );
System.getProperties().put( "proxyPort", config.get_proxyPort() );

If I set the httpClient proxy parameters directly instead of =E2=80=9Cserve=
r failed
to respond=E2=80=9D exception I get this exception: =E2=80=9CHTTPclinet.fet=
ch Method failed:
HTTP/1.1 502 Proxy Error (The specified network name is no longer available=
.=20
)=E2=80=9D.

When this occurs even the Internet Explorer fails to fetch the specified UR=
I
(e.g.
http://z3950.loc.gov:7090/voyager?version=3D1.1&operation=3DsearchRetrieve&=
maximumRecords=3D1000&query=3Dtitle=3D%22Nieuw+Archief+voor+Wiskunde%22
),=20

however it still works with other URIs like Google.

To prevent this problem I have to send about 10 queries wait for 10 seconds
and then send another 10 queries.  Using this setting I ran about 20,000
queries overnight successfully.

The most puzzling is that if I do not change the URL and send the same quer=
y
over and over again no exception happens.

At this point I only can think of two things happening here:

1. My proxy server does not respond after a while because it thinks I am an
internal DOS attacker.

2. The server thinks I am a DOS attacker and does not respond if I send the
queries to fast.

I would appreciate your suggestions.

Notes:

1. I do release the connections after each use.
2. I have tried sending queries with the connection close header.
3. I also tried another setting where I run 10 queries close/release
everything shutdown the connection manager and create a new one and send
another 10 queries.  =20

Thanks,
Arash Joorabchi

--=20
View this message in context: http://www.nabble.com/problem-with-multithrea=
ding-and-proxy-tp21912219p21912219.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org