hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Radim Kolar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4851) add lifecycle to Comparators
Date Thu, 06 Dec 2012 16:37:09 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511499#comment-13511499
] 

Radim Kolar commented on MAPREDUCE-4851:
----------------------------------------

for example you can not use datagrid in comparator because if you start it in setConf() you
can shut it down, which is kinda huge memory leak if JVM reuse is enabled. It will also block
JVM exit unless hadoop is using System.exit() call.
                
> add lifecycle to Comparators
> ----------------------------
>
>                 Key: MAPREDUCE-4851
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4851
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Radim Kolar
>              Labels: mrv2
>
> current mapreduce api is using RawComparator interface in:
> setGroupingComparatorClass
> setSortComparatorClass
> This interface has no lifecycle support. I propose to change that methods to take argument
new class implements RawComparator with setup and cleanup methods. 
> This will leave existing code with RawComparator alone, Providing some backward compatibility.
> new class: 
> class SortComparator implements RawComparator

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message