hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From olze <oliver.zem...@gmail.com>
Subject Threads hang in ThreadSafeClientConnManager.getConnection:216
Date Wed, 19 Jan 2011 10:32:51 GMT

I wrote a multithreaded Tool which creates some DefaultHttpClient instances
and HttpGets which then download a site (like a crawler). My Problem is
that, after a couple of minutes (~20) the application goes slower and
slower. In the beginning 10 Threads are running, and after about 20 minutes
only one single thread is running. So i took a look at the debugging window
in netbeans and saw that all other threads are hanging in
ThreadSafeClientConnManager.getConnection:216

i made a screenshot with the thread trace:
http://www.imagebanana.com/view/zljfdwh4/screenshot.PNG

I already changed the default max. connections per route to 20.

the source i use (initialisation):
schemeRegistry = new SchemeRegistry();
schemeRegistry.register(
     new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(
     new Scheme("https", 443, SSLSocketFactory.getSocketFactory()));

cm = new ThreadSafeClientConnManager(schemeRegistry);
cm.closeIdleConnections(4, TimeUnit.SECONDS);
cm.closeExpiredConnections();
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(100);

the usage (threads):
DefaultHttpClient client = new DefaultHttpClient(Main.cm, Main.params);
client.setRedirectHandler(new CustomRedirectHandler());
client.getParams().setParameter(ClientPNames.COOKIE_POLICY,
            
org.apache.http.client.params.CookiePolicy.BROWSER_COMPATIBILITY);

HttpResponse response = null;

try {
    response = client.execute(get);
} catch (Exception ex) {
     get.abort();
}

entity = response.getEntity();
if (entity == null) {
    get.abort();
    pageURL = null;
    client = null;
    return;
}

fos = new FileOutputStream(new File(filename));
try {
       entity.writeTo(fos);
     } catch (Exception ex) {
       ex.printStackTrace();
       if (entity != null) {
             EntityUtils.consume(entity);
       }
}

this is not the complete code, i just copied the important parts. i always
use EntityUtils.consume() and get.abort if i see something unexpected
happened, so i think all resources should be free'd?

Is there any way i can check if the resources are 100% free'd or if there is
some leak?

Thanks in advance.
-- 
View this message in context: http://old.nabble.com/Threads-hang-in-ThreadSafeClientConnManager.getConnection%3A216-tp30706069p30706069.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message