hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rob Griffin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HTTPASYNC-116) Cancelling the Future returned by CloseableHttpAsyncClient should release all resources allocated to the request that created it.
Date Fri, 27 Jan 2017 02:30:24 GMT

    [ https://issues.apache.org/jira/browse/HTTPASYNC-116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840861#comment-15840861

Rob Griffin commented on HTTPASYNC-116:


I have finally got around to building the library a few days ago and since then I have had
another out of memory problem. An analysis of the memory dump shows that all nested future
tasks are now correctly flagged as cancelled however the CPool object in org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager
has 209,487 item  org.apache.http.nio.pool.LeaseRequest objects in it's linked list. It seems
to me that cancellation does not remove the LeaseRequest from the connection manager.

I've attached a couple of screen dumps to show what I saw. I have a memory dump (.hprof file)
if you want to examine it. It's a bit over 4GB in size. 

> Cancelling the Future returned by CloseableHttpAsyncClient should release all resources
allocated to the request that created it.
> ---------------------------------------------------------------------------------------------------------------------------------
>                 Key: HTTPASYNC-116
>                 URL: https://issues.apache.org/jira/browse/HTTPASYNC-116
>             Project: HttpComponents HttpAsyncClient
>          Issue Type: Improvement
>    Affects Versions: 4.1.1
>            Reporter: Rob Griffin
>            Assignee: Oleg Kalnichevski
>             Fix For: 4.1.2
>         Attachments: ScreenShot001.jpg, ScreenShot026.jpg
> We call CloseableHttpAsyncClient.execute() to execute HTTP PUTs at the rate of several
hundred per minute. Sometimes our web site slows down and does not respond quickly enough
and when this occurs the requests back up. We have code that detects this and cancels the
Future returned from the execute method when the request has waited too long. If this happens
too often the application crashes with an out of memory error. 
> Analysis of a dump showed that there were more 108,000 instances of org.apache.http.nio.pool.LeaseRequest
along with a similar number of instances of other HTTP Client classes. Inspecting one of these
objects showed that its future variable is not cancelled but that by tracing though the callback
variables there is a cancelled Future further up the chain. That cancelled Future object is
one returned by execute because its callback is one of our classes. So it appears that the
library is unaware that cancel has been called on the Future returned by execute() and keeps
a reference to all resources allocated to the request.
> Instead cancelling the Future should release any resources allocated to the request and
should also cancel execution if it has started.
> We regard this as a very urgent issue because it causes our application to crash. We
realise that improving the responsiveness of our web site will alleviate the problem somewhat
but the library shouldn't leak memory. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org

View raw message