hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-1183) Serializable job components: Mapper, Reducer, InputFormat, OutputFormat et al
Date Thu, 05 Nov 2009 02:09:32 GMT
Serializable job components: Mapper, Reducer, InputFormat, OutputFormat et al
-----------------------------------------------------------------------------

                 Key: MAPREDUCE-1183
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1183
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: client
    Affects Versions: 0.21.0
            Reporter: Arun C Murthy
            Assignee: Arun C Murthy


Currently the Map-Reduce framework uses Configuration to pass information about the various
aspects of a job such as Mapper, Reducer, InputFormat, OutputFormat, OutputCommitter etc.
and application developers use org.apache.hadoop.mapreduce.Job.set*Class apis to set them
at job-submission time:

{noformat}
Job.setMapperClass(IdentityMapper.class);
Job.setReducerClass(IdentityReducer.class);
Job.setInputFormatClass(TextInputFormat.class);
Job.setOutputFormatClass(TextOutputFormat.class);
...
{noformat}

The proposal is that we move to a model where end-users interact with org.apache.hadoop.mapreduce.Job
via actual objects which are then serialized by the framework:
{noformat}
Job.setMapper(new IdentityMapper());
Job.setReducer(new IdentityReducer());
Job.setInputFormat(new TextInputFormat("in"));
Job.setOutputFormat(new TextOutputFormat("out"));
...
{noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message