hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Ratan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1986) Add support for a general serialization mechanism for Map Reduce
Date Mon, 29 Oct 2007 08:24:52 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538429

Vivek Ratan commented on HADOOP-1986:

But I *was* talking specifically about RecordSerializer, and in the context of what interface
we should have to handle serializers that create their own objects and those that take in
a reference. My comments were in response to Doug's example. To paraphrase my argument, having
a serializer keep track of the class it was created for is difficult/restrictive if the serialize
can handle more than one class. 

Yes, serializers can handle arbitrary constructors, but a serializer than can deserialize
more than one class (for example, one for Record I/O that can deserialize any class that inherits
from the base Record I/O class) requires that the classes all be constructed the same way
(i.e. have the same constructor signature), if it is responsible for creating deserialized
objects. Or else you give it an already-constructed object (and it just fills in the fields
when deserializing). 

> Add support for a general serialization mechanism for Map Reduce
> ----------------------------------------------------------------
>                 Key: HADOOP-1986
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1986
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Tom White
>            Assignee: Tom White
>             Fix For: 0.16.0
>         Attachments: SerializableWritable.java, serializer-v1.patch
> Currently Map Reduce programs have to use WritableComparable-Writable key-value pairs.
While it's possible to write Writable wrappers for other serialization frameworks (such as
Thrift), this is not very convenient: it would be nicer to be able to use arbitrary types
directly, without explicit wrapping and unwrapping.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message