Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Thu, 26 Sep 2013 15:52:02 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.12670750.1380209873068.24848.1380210722866@arcas>
In-Reply-To: <JIRA.12670750.1380209873068@arcas>
References: <JIRA.12670750.1380209873068@arcas>
Subject: [jira] [Commented] (MAPREDUCE-5543) In-memory map outputs can be
 leaked after shuffle completes in 0.23
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-5543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778922#comment-13778922 ] 

Hadoop QA commented on MAPREDUCE-5543:
--------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12605266/MAPREDUCE-5493-branch-0.23.patch
  against trunk revision .

    {color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4051//console

This message is automatically generated.
                
> In-memory map outputs can be leaked after shuffle completes in 0.23
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5543
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5543
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.9
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Blocker
>         Attachments: MAPREDUCE-5493-branch-0.23.patch
>
>
> MergeManagerImpl#close adds the contents of inMemoryMergedMapOutputs and inMemoryMapOutputs to a list of map outputs that is subsequently processed, but it does not clear those sets.  This prevents some of the map outputs from being garbage collected and significantly reduces the memory available for the subsequent reduce phase.
> This was fixed for trunk and branch-2 by MAPREDUCE-5493, but that has since been closed after 2.1.1 released.  This JIRA tracks backporting the fix to branch-0.23 as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira