hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6685) Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
Date Mon, 22 Nov 2010 18:06:20 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12934534#action_12934534
] 

Owen O'Malley commented on HADOOP-6685:
---------------------------------------

{quote}
We should refrain from adding any new data formats to the Hadoop kernel. More generally, we
should refrain from adding code that could be implemented as user code to the kernel.
{quote}

This is a major change in the direction of Hadoop that has never been discussed by the Hadoop
PMC. You are welcome to start such a thread on general@, but until the Hadoop PMC approves
it, it is *not* the plan of record.

SequenceFile, far from being deprecated, stores the majority of the world's Hadoop information.
There are no current replacements for its functionality and implicitly deprecating it would
be very expensive. Without the ability to extend SequenceFile to use the new generic serialization
api, users will have no way of using the new api for storing their data. 

The part of the bylaws (http://bit.ly/9olNWB ) on vetoes say that if the validity of the veto
is questioned, it must be confirmed by someone with a binding vote. Does someone want to stand
up and confirm the veto's validity?

> Change the generic serialization framework API to use serialization-specific bytes instead
of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: libthrift.jar, serial.patch, serial4.patch, serial6.patch, serial7.patch,
SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the
serialization specific configuration. Since this data is really internal to the specific serialization,
I think we should change it to be an opaque binary blob. This will simplify the interface
for defining specific serializations for different contexts (MAPREDUCE-1462). It will also
move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message