hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fang Lin <Fang...@u.washington.edu>
Subject Only create 2 connection per route
Date Wed, 27 Jul 2011 23:46:02 GMT
I am now puzzled by the following two cases with my ThreadSafeClientConnManager :
1.  getConnectionsInPool (route) call shows that the value increased from 1 to 2 quickly,
but never goes beyond 2.
2.  getConnectionsInPool (route) always return zero for the https routes.
Any ideas?

I did the following:
1. setDefaultMaxPerRoute to 5;  the getDefaultMaxPerRoute call confirmed it.
2. Also set individual route's max connections via setMaxForRoute (route) call;  getMaxForRoute
(route) confirmed the value was set correctly.

Did the following for response handling:
1. If exception occurs, abort the current http get request
2. Otherwise, make sure the response entity is consumed in both successful response and response
with an error code

The code of my ConnPoolManager class:

public class ConnPoolManager {
  private final static Logger log =   LoggerFactory.getLogger("ConnPoolManager");
  private static ContentEncodingHttpClient client;
  private static ThreadSafeClientConnManager manager;

  static {
    try {
       SchemeRegistry registry = new SchemeRegistry();
      registry.register(new Scheme("http", 
				   PlainSocketFactory.getSocketFactory(), 
				   80));
      registry.register(new Scheme("https", 
				   new MySocketFactory(),
				   443));

      HttpParams params = new BasicHttpParams();
      HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
      HttpProtocolParams.setContentCharset(params, "UTF-8");
      HttpProtocolParams.setUseExpectContinue(params, true);
      
      params.setBooleanParameter
	(CoreProtocolPNames.USE_EXPECT_CONTINUE, false);
      params.setBooleanParameter
	(CoreConnectionPNames.STALE_CONNECTION_CHECK, false);
      params.setIntParameter
	(CoreConnectionPNames.CONNECTION_TIMEOUT, 
	 PoolProperties.CONNECTION_TIMEOUT);
      params.setIntParameter
	(CoreConnectionPNames.SO_TIMEOUT, 
	 PoolProperties.SOCKET_TIMEOUT);
      params.setBooleanParameter
	(CoreConnectionPNames.SO_REUSEADDR, true);
      params.setIntParameter
	(CoreConnectionPNames.SOCKET_BUFFER_SIZE, 
	 PoolProperties.SOCKET_BUFFER_SIZE);
      params.setIntParameter(ClientPNames.MAX_REDIRECTS, 1);

      manager = new ThreadSafeClientConnManager (params, registry);
      manager.setMaxTotal (PoolProperties.MAX_POOL_SIZE);
      manager.setDefaultMaxPerRoute (PoolProperties.MAX_PER_ROUTE);

      client = new ContentEncodingHttpClient (manager, params);
      client.setReuseStrategy (new ConnectionReuseStrategy() {
	  public boolean keepAlive(HttpResponse response,
				   HttpContext context) {
	    return true;
	  }
	});
      client.setKeepAliveStrategy (new ConnectionKeepAliveStrategy() {
	  public long getKeepAliveDuration(HttpResponse response, 
					       HttpContext context) {
	    return PoolProperties.KEEP_ALIVE_DURATION;
	  }
	});
    } catch (Exception e) {
      log.error ("Failed to set up ConnPoolManager", e);
    }
  }
				
  private ConnPoolManager () { }
  
 public static void setMaxForRoute (HttpRoute route,
				        int maxConn)
  {
      manager.setMaxForRoute (route, maxConn);
  }

  public static int getMaxForRoute (HttpRoute route)
  {
    return manager.getMaxForRoute (route);
  }

  public static int getConnectionsInPool (HttpRoute route)
  {
    return manager.getConnectionsInPool (route);
  }

  public static int getConnectionsInPool ()
  {
    return manager.getConnectionsInPool ();
  }

  public static int getMaxTotal ()
  {
    return manager.getMaxTotal();
  }

  public static HttpResponse execute (HttpRoute route,
				           HttpUriRequest request,
				           HttpContext localContext)
    throws IOException
  {
    return (localContext == null
	    ? client.execute(route.getTargetHost(), request)
	    : client.execute(route.getTargetHost(), request, localContext));
  }

  public static void logCon (HttpRoute route)
  {
    log.info (route.getTargetHost().toURI() + " has " +
	      getConnectionsInPool  (route) + "/" +
	      getMaxForRoute (route) +
	      " in-use/max of this route.");
    logStatus();
  }

  private static String poolStatus()
  {
    return ("Pool has total active httpclient connections/max: " + 
	    getConnectionsInPool() + "/" + getMaxTotal ());
  }
  public static void logStatus()
  {
    log.info (poolStatus ());
  }

  public static void shutdown()
  {
    manager.shutdown();
  }
}
-----Original Message-----
From: Sam Crawford [mailto:samcrawford@gmail.com] 
Sent: Friday, July 22, 2011 5:55 AM
To: HttpClient User Discussion
Subject: Re: ConnectionPoolTimeoutException

My experience leads me to agree with Oleg. I would suggest adding logging for connection management
(see http://hc.apache.org/httpcomponents-client-ga/logging.html).

Make sure you're always consuming the content when handling responses.
When I first began working with HttpClient I was aborting our response handling process if
I saw a non-2xx/3xx response code *without* consuming the response - this led to the connection
not being evicted.

Thanks,

Sam


On 22 July 2011 13:26, Oleg Kalnichevski <olegk@apache.org> wrote:
> On Thu, 2011-07-21 at 18:47 +0000, Fang Lin wrote:
>> Often getting org.apache.http.conn.ConnectionPoolTimeoutException: 
>> Timeout waiting for connection
>>         at 
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(Conn
>> PoolByRoute.java:417)
>>         at 
>> org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPo
>> olByRoute.java:300)
>>         at 
>> org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConn
>> ection(ThreadSafeClientConnManager.java:224)
>>         at 
>> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq
>> uestDirector.java:391)
>>         at 
>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>> ient.java:820) when there were only two active connections in a 
>> ConnPoolByRoute whose  max connection is set to 10.
>> This case happened with some routes but not all routes. Once it started, only restarting
tomcat would fix the issue.
>> Any suggestion?
>>
>>
>> Httpclient 4.1.1, httpcore 4.1.2
>>
>> Java(TM) SE Runtime Environment (build 1.6.0_23-b05) Java HotSpot(TM) 
>> 64-Bit Server VM (build 19.0-b09, mixed mode)
>>
>
> I suspect your code is leaking connections.
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message