hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: AbstractNIOConnPool.processPendingRequests is never called?
Date Thu, 28 Nov 2013 09:18:06 GMT
On Wed, 2013-11-27 at 22:09 +0400, Dmitry Potapov wrote:
> Hello everyone,
> 
> I've got OOM today and trying to understand if I'm using HttpAsyncClient wrong.
> 
> So. I have HttpAsyncClient sending 100-150 requests per second to single remote
> host. Connection request timeout is 50 ms, connect timeout is 25 ms, socket
> timeout is 100 ms.
> In heap dump I've found about 200,000 LeaseRequest object. Since
> completedRequests is cleaned in the end of each lease(...) function call, I'm
> pretty sure that this is a leasingRequests.
> 
> Looks like that remote host went down for a while and stopped requests
> processing. So, CPool started to accumulate lease requests and after several
> hours tons of accumulated LeaseRequest objects caused OOM.
> I've tried to find where leasingRequests must be cleaned and found that only
> function which performs cleanup by request deadline is
> AbstractNIOConnPool.processPendingRequests() which is called only by
> PoolingNHttpClientConnectionManager.closeIdleConnections() or
> .closeExpiredConnections().
> 
> So, my question is, should I call this functions periodically on my side, or
> there is another leasingRequests clean up mechanism which I didn't take into
> account?
> 

Dmitry,

The only way to force the pooling connection manager to do a full linear
scan on the connection request queue is by calling
#closeExpiredConnections every once in a while (usually from a monitor
thread).

However, the queue _should_ not get clogged with requests as long as
both connect and connection request timeouts are set to a positive
value. A connect timeout always causes eviction of expired requests from
the queue until a non-expired request is found. This strategy generally
should be good enough. 

Would it be possible for you to reproduce the issue with the context
logging for connection management / request execution turned on?

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message