hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1986) Add support for a general serialization mechanism for Map Reduce
Date Wed, 02 Apr 2008 22:28:24 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Tom White updated HADOOP-1986:

    Release Note: 
Programs that implement the raw Mapper or Reducer interfaces will need modification to compile
with this release. For example, 

class MyMapper implements Mapper {
  public void map(WritableComparable key, Writable val,
    OutputCollector out, Reporter reporter) throws IOException {
    // ...
  // ...

will need to be changed to refer to the parameterized type. For example:

class MyMapper implements Mapper<WritableComparable, Writable, WritableComparable, Writable>
  public void map(WritableComparable key, Writable val,
    OutputCollector<WritableComparable, Writable> out, Reporter reporter) throws IOException
    // ...
  // ...

Similarly implementations of the following raw interfaces will need modification: InputFormat,
OutputCollector, OutputFormat, Partitioner, RecordReader, RecordWriter 
    Hadoop Flags: [Incompatible change, Reviewed]  (was: [Reviewed, Incompatible change])

> Add support for a general serialization mechanism for Map Reduce
> ----------------------------------------------------------------
>                 Key: HADOOP-1986
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1986
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Tom White
>            Assignee: Tom White
>             Fix For: 0.17.0
>         Attachments: hadoop-serializer-v2.tar.gz, SequenceFileWriterBenchmark.java, SerializableWritable.java,
serializer-v1.patch, serializer-v2.patch, serializer-v3.patch, serializer-v4.patch, serializer-v5.patch,
serializer-v6.patch, serializer-v7.patch
> Currently Map Reduce programs have to use WritableComparable-Writable key-value pairs.
While it's possible to write Writable wrappers for other serialization frameworks (such as
Thrift), this is not very convenient: it would be nicer to be able to use arbitrary types
directly, without explicit wrapping and unwrapping.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message