hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francois-Xavier Bonnet (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HTTPCLIENT-1263) CachingHttpClient not consuming backend HttpResponse entity causing PoolingClientConnectionManager to become unresponsive
Date Thu, 22 Nov 2012 16:00:58 GMT

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502834#comment-13502834
] 

Francois-Xavier Bonnet commented on HTTPCLIENT-1263:
----------------------------------------------------

I just tried to do some debugging. It seems that the problem is due to the combination of
CachingHttpClient with BasicHttpCache and DecompressingHttpClient.
SizeLimitedResponseReader is never closing the InputStream assuming that it will be closed
automatically and release the connection thanks to ContentLengthInputStream and EofSensorWatcher.
But when you are using a DecompressingHttpClient, Content-length header is removed.

I suggest that we close the InputStream in method org.apache.http.impl.client.cache.SizeLimitedResponseReader.doConsume()
if the limit has not been reached.
I can work on a patch with unit tests if you want.
                
> CachingHttpClient not consuming backend HttpResponse entity causing PoolingClientConnectionManager
to become unresponsive
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HTTPCLIENT-1263
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1263
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.2.1
>            Reporter: Bryant Harris
>              Labels: cache, connection-pooling, consumed
>
> I've noticed that when issuing requests via a pooled and cached HttpClient that the client
eventually becomes unresponsive (which appears to be because an HttpEntity is not getting
consumed properly).
> Steps to reproduce.
> 1. Here is how I've configured all the relevant classes.
>     HttpClient standardClient = null;
>     HttpClient cachedClient = null;
>     PoolingClientConnectionManager connectionManager = null;
>     protected synchronized HttpClient getStandardClient() {
>       if ( standardClient == null ) {
>         connectionManager = new PoolingClientConnectionManager();
>         connectionManager.setMaxTotal(2);
>         connectionManager.closeIdleConnections(120, TimeUnit.SECONDS);
>         standardClient = new DecompressingHttpClient( new DefaultHttpClient (connectionManager));
>       }
>       return standardClient;
>     }
> protected synchronized HttpClient getCachedClient() {
>   if ( cachedClient == null ) {
>     CacheConfig cacheConfig = new CacheConfig();
>     cacheConfig.setMaxObjectSize( 512*1024 );
>     cacheConfig.setMaxCacheEntries( 10 );
>     cachedClient = new CachingHttpClient(getStandardClient(),
>                                          getCacheStorage(),
>                                          cacheConfig);
>   }
>   return cachedClient;
> }
> As you can see I have two http clients. A caching http client that wraps the standard
client.
> Now what I've found is that if I remove cachedClient and only use standardClient, I don't
have any issues with the pool hanging and orphaned connections.
> 2.  Here is my code for how I issue and consume requests
>     HttpClient httpClient = cacheOkay ? getCachedClient() : getStandardClient();  
>     HttpResponse response = httpClient.execute(request, localContext);
>     HttpEntity resEntity = response.getEntity();  
>     int responseStatus = response.getStatusLine().getStatusCode();
>     byte[] responseBody = EntityUtils.toByteArray(resEntity);
>     EntityUtils.consume(resEntity);
> If you set up a test like this and use the cached client, it will hang fairly quickly.
> I've been able to work around this by creating the CachingHttpClient as follows:
>     protected synchronized HttpClient getCachedClient() {
>     	if ( cachedClient == null ) {
>     		WhosHereApplication application = WhosHereApplication.getInstance();
>     		cachedClient = new CachingHttpClient(getStandardClient(),
>     											 new HeapResourceFactory() {
> 													@Override
> 													public Resource generate(
> 															String requestId,
> 															InputStream instream,
> 															InputLimit limit)
> 															throws IOException {
> 														try {
> 															return super.generate(requestId, instream, limit);
> 														}
> 														finally {
> 															instream.close();
> 														}
> 													}
>     			
>     											 },
>     				                             application.getCacheStorage(),
>     				                             application.getCacheConfig());
>     		Log.i(tag, "Creating CachingHttpClient");
>     	}
>     		
>     	return cachedClient;
>     }
> Notice the inline subclass of HeapResourceFactory where I add the stream close call.
 Once I add this the caching client no longer freezes up.
> I'm not familiar enough with the source code to pinpoint the issue, but appears the back
end entity is not getting consumed properly, forcing this work around.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message