hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5974) Allow map output collector fallback
Date Wed, 20 Aug 2014 22:01:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104674#comment-14104674

Karthik Kambatla commented on MAPREDUCE-5974:

+1. Will commit this later today, if no one objects. 

> Allow map output collector fallback
> -----------------------------------
>                 Key: MAPREDUCE-5974
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5974
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: task
>    Affects Versions: 2.6.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: mapreduce-5974.txt
> Currently we only allow specifying a single MapOutputCollector implementation class in
a job. It would be nice to allow a comma-separated list of classes: we should try each collector
implementation in the user-specified order until we find one that can be successfully instantiated
and initted.
> This is useful for cases where a particular optimized collector implementation cannot
operate on all key/value types, or requires native code. The cluster administrator can configure
the cluster to try to use the optimized collector and fall back to the default collector.

This message was sent by Atlassian JIRA

View raw message