Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Tue, 30 Oct 2012 16:06:12 +0000 (UTC)
From: "Robert Joseph Evans (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <947169655.45027.1351613172926.JavaMail.jiratomcat@arcas>
In-Reply-To: <757108863.33796.1351289112614.JavaMail.jiratomcat@arcas>
Subject: [jira] [Updated] (HADOOP-8986) Server$Call object is never released
 after it is sent
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Joseph Evans updated HADOOP-8986:
----------------------------------------

       Resolution: Fixed
    Fix Version/s: 0.23.5
                   2.0.3-alpha
                   3.0.0
           Status: Resolved  (was: Patch Available)

Thanks Daryn and Suresh for the reviews. 

I waited to check it in until I could run one more test to measure how much memory savings this had on my original test case.  I ran a sleep job with 20,000 mappers and 3,000 reducers.  This was on a cluster large enough and free enough that all of them could be running at the same time.  The peek memory usage decreased from about 500MB to 285MB and the final heap as most of the reducers started to get close to completing dropped to 200MB, where as before it was still up at 500MB.

I put it into trunk, branch-2, and branch-0.23
                
> Server$Call object is never released after it is sent
> -----------------------------------------------------
>
>                 Key: HADOOP-8986
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8986
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: ipc
>    Affects Versions: 2.0.2-alpha, 0.23.4
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>            Priority: Critical
>             Fix For: 3.0.0, 2.0.3-alpha, 0.23.5
>
>         Attachments: HADOOP-8986.txt
>
>
> When an IPC response cannot be returned without blocking an Server$Call object is attached to the SelectionKey of the write selector.  However the call object is never removed from the SlectionKey. So for a connection that rarely has large responses but is long lived there is a lot of data.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira