hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6685) Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
Date Mon, 29 Nov 2010 17:31:26 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12964847#action_12964847
] 

Tom White commented on HADOOP-6685:
-----------------------------------

Owen> All of the technical feedback for this patch has been addressed, including technical
feedback that Tom gave me offline.

Was this the feedback I gave on MAPREDUCE-1462 back in February? I haven't given any feedback
offline for this issue.

Owen> Making compromises to not change SequenceFile or moving the plugins to the contrib
module that would destroy the usability of the patch, isn't really compromise. They are just
thinly veiled attempts to make it difficult to use ProtoBufs or Thrift in user's applications.

The original work for Thrift and Protocol Buffers serializations (MAPREDUCE-376 and MAPREDUCE-377)
was as contrib modules, so if we want to change that approach, then we need to get consensus
on doing so. That consensus hasn't been forthcoming so they should be left as optional contrib
modules. Serializations in this form are easy to use by the way: users just add the relevant
contrib jar and the serialization jar to the job, just like any other dependency.



> Change the generic serialization framework API to use serialization-specific bytes instead
of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: libthrift.jar, serial.patch, serial4.patch, serial6.patch, serial7.patch,
SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the
serialization specific configuration. Since this data is really internal to the specific serialization,
I think we should change it to be an opaque binary blob. This will simplify the interface
for defining specific serializations for different contexts (MAPREDUCE-1462). It will also
move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message