hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-8986) Server$Call object is never released after it is sent
Date Tue, 30 Oct 2012 16:06:12 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Robert Joseph Evans updated HADOOP-8986:

       Resolution: Fixed
    Fix Version/s: 0.23.5
           Status: Resolved  (was: Patch Available)

Thanks Daryn and Suresh for the reviews. 

I waited to check it in until I could run one more test to measure how much memory savings
this had on my original test case.  I ran a sleep job with 20,000 mappers and 3,000 reducers.
 This was on a cluster large enough and free enough that all of them could be running at the
same time.  The peek memory usage decreased from about 500MB to 285MB and the final heap as
most of the reducers started to get close to completing dropped to 200MB, where as before
it was still up at 500MB.

I put it into trunk, branch-2, and branch-0.23
> Server$Call object is never released after it is sent
> -----------------------------------------------------
>                 Key: HADOOP-8986
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8986
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: ipc
>    Affects Versions: 2.0.2-alpha, 0.23.4
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>            Priority: Critical
>             Fix For: 3.0.0, 2.0.3-alpha, 0.23.5
>         Attachments: HADOOP-8986.txt
> When an IPC response cannot be returned without blocking an Server$Call object is attached
to the SelectionKey of the write selector.  However the call object is never removed from
the SlectionKey. So for a connection that rarely has large responses but is long lived there
is a lot of data.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message