hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1986) Add support for a general serialization mechanism for Map Reduce
Date Thu, 18 Oct 2007 21:18:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12536053
] 

Doug Cutting commented on HADOOP-1986:
--------------------------------------

Also, instead of using introspection, we might use a method, e.g.:

{noformat}
public interface Serialization {
  boolean accept(Class c);
  Serializer getSerializer();
  Deserializer getDeserializer();
}
{noformat}

SerializationFactory#getSerialization(Class) would then just iterate through the defined serializations
calling accept().  It could cache an instance of each defined serialization.  Note also that,
for primitive types, we could pass things like Integer.TYPE.  So then one might even, e.g.,
define a mapper that takes <int,long> pairs.


> Add support for a general serialization mechanism for Map Reduce
> ----------------------------------------------------------------
>
>                 Key: HADOOP-1986
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1986
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Tom White
>            Assignee: Tom White
>             Fix For: 0.16.0
>
>         Attachments: SerializableWritable.java, serializer-v1.patch
>
>
> Currently Map Reduce programs have to use WritableComparable-Writable key-value pairs.
While it's possible to write Writable wrappers for other serialization frameworks (such as
Thrift), this is not very convenient: it would be nicer to be able to use arbitrary types
directly, without explicit wrapping and unwrapping.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message