hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6685) Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
Date Sat, 20 Nov 2010 03:34:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12934094#action_12934094
] 

Owen O'Malley commented on HADOOP-6685:
---------------------------------------

I question the validity of Doug's veto. His objection to the patch has nothing to do with
the merits of the patch and everything to do with his wish to push Avro into Hadoop at the
cost of the users.


{quote}
it didn't extend SequenceFile
{quote}

Why is no extension of SequenceFile acceptable? Would it be valid if someone on subversion
said that no further enhancements to one of the primary repository access modules was acceptable?
It is unacceptable to prevent future enhancements of a critical part of Hadoop's library code.

{quote}
it didn't create a core dependency on ProtocolBuffers
{quote}

Why is depending on Avro in the framework acceptable, but not ProtocolBuffers? I could have
implemented the versioning by hand, but that is ridiculous. ProtocolBuffers provides exactly
the capabilities that I need. Avro does not. I've discussed why above. The projects are not
interchangeable. I understand that you have been working almost exclusively on Avro for the
last 2 years, but that does not mean that Hadoop must use Avro exclusively.



> Change the generic serialization framework API to use serialization-specific bytes instead
of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: libthrift.jar, serial.patch, serial4.patch, serial6.patch, serial7.patch,
SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the
serialization specific configuration. Since this data is really internal to the specific serialization,
I think we should change it to be an opaque binary blob. This will simplify the interface
for defining specific serializations for different contexts (MAPREDUCE-1462). It will also
move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message