hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6685) Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
Date Sat, 27 Nov 2010 01:31:20 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12936141#action_12936141
] 

Chris Douglas commented on HADOOP-6685:
---------------------------------------

Nigel> There's some truth in what you say (I qualified it as "part of" the problem), but
this is the wrong forum. There exist technical disagreements impeding this particular issue.
To identify and resolve them, participants must focus on the code and on none of the items
you raise. Please start a thread on private@ if you want to discuss personnel issues, where
you can be more explicit about your concerns.

Doug> On HADOOP-6659: Eric doesn't have a veto. Jeff was the first to ask for a design
document. This issue adds not only the APIs, but- as Owen explained- it includes changes to
SequenceFile to illustrate its completeness to reviewers. This approach, as agreed in the
contributor meetings, was to avoid outcomes like MAPREDUCE-1126 where large changes trickle
in over months, making review and participation impossible. Upfront design and consensus followed
by more complete patches was supposed to enable progress. For a few days, that was in place
here.

You did not respond to my proposal. Can we cling to differences on the serialization format
of serializer metadata as the point at issue? Agreement on project direction/scope, programming
taste, and packaging can be hashed out elsewhere. Adding such large, contentious issues as
prerequisites ensures failure. Factoring out file formats into external projects and improving
the way dependencies are packaged in Hadoop could proceed even if this patch were committed.

> Change the generic serialization framework API to use serialization-specific bytes instead
of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: libthrift.jar, serial.patch, serial4.patch, serial6.patch, serial7.patch,
SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the
serialization specific configuration. Since this data is really internal to the specific serialization,
I think we should change it to be an opaque binary blob. This will simplify the interface
for defining specific serializations for different contexts (MAPREDUCE-1462). It will also
move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message