hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Fix <josh...@yahoo.com>
Subject Using MultiThreadedHttpConnectionManager in a Servlet
Date Wed, 26 Sep 2007 19:55:49 GMT
I have a servlet that reads some parameters from a URL, uses an HttpURLConnection to query
and retrieve a web page, parse the text, and finally construct some XML as output.  I know
using an HTTP connection as the primary back-end data source is horribly bad, but we have
to do it for now and the web pages it is scraping are all internal to our large company.

The remote server sometimes takes a long time to respond.  The HttpURLConnection/BufferedReader
methods seem to act as if they are synchronized--only one connection/request can be made at
a time.  This can cause significant congestion when multiple requests come in at once.

My goal was to use HttpClient with the MultiThreadedHttpConnectionManager to handle all of
these back-end connections, but so far it seems like the behavior is the same.  Is it possible
to have multiple threads from my servlet open simultaneous connections to an external web
site with different requests and read the results?  I would really like to get around the
blocking aspect of the read statements.  My servlet declares the MTHCM as a static object
in the servlet class, so that once instance is shared amongst all incoming servlet requests,
and a static{} block is used in the class to set timeouts/etc.  The HttpClient is newed in
the doGet method.

If I build a test servlet that does nothing but return a string after about 20 seconds to
simulate the remote server responding slowly and point my real servlet to the test servlet,
no other requests/threads to the real servlet are allowed to make a connection until the test
servlet returns it's result.  What am I doing wrong?  How can I allow other threads to continue
to query the server, even when one request is very slow to respond?  Thank you in advance
for you help!


Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message