hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jothi Padmanabhan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5493) Shuffle copiers do not return Codecs back to the pool at the end of shuffling
Date Mon, 16 Mar 2009 11:14:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jothi Padmanabhan updated HADOOP-5493:
--------------------------------------

    Status: Patch Available  (was: Open)

Test Patch results. Writing test case for this is non trivial. Ideally we should have counters
from the codec pool that we can both expose outside and also use in test cases to check if
the compressors and decompressors are being returned back correctly.
 
    [exec] -1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     -1 tests included.  The patch doesn't appear to include any new or modified
tests.
     [exec]                         Please justify why no tests are needed for this patch.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
     [exec] 
     [exec]     +1 release audit.  The applied patch does not increase the total number of
release audit warnings.
     [exec] 

> Shuffle copiers do not return Codecs back to the pool at the end of shuffling
> -----------------------------------------------------------------------------
>
>                 Key: HADOOP-5493
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5493
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Devaraj Das
>            Assignee: Jothi Padmanabhan
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: hadoop-5493.patch
>
>
> At the end of shuffle, the copiers should return back the codecs to the pool. This doesn't
happen and can potentially lead to a lot of memory leak on the reduce task (depending on how
many shuffle copiers there are).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message