hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6685) Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
Date Mon, 06 Dec 2010 19:19:19 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968373#action_12968373
] 

Doug Cutting commented on HADOOP-6685:
--------------------------------------

> The API is binary.

The API is not binary today.  You propose a change to binary, but have not motivated that
change in a convincing manner.

> Designing an API to make efficient implementation impossible is an anti-goal.

There is no evidence that serialization metadata is performance critical.  Even if it were,
there is evidence that textual formats like JSON can be processed at comparable speeds to
binary.  Textual formats are useful for configuration data.  Supporting multiple formats is
error-prone and introduces needless complexity.  We should support a minimal number of formats
for configuration data to reduce confusion.  To date, the external format has been XML and
the internal representation has been Map<String,String>.  If you wish to improve this
you need to provide a compelling argument.  Lack of support for nesting and strong typing
are accepted weaknesses of the current approach.  These might be improved by JSON, YAML or
a different XML format combined with a bean-like internal representation.

> Change the generic serialization framework API to use serialization-specific bytes instead
of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: serial.patch, serial4.patch, serial6.patch, serial7.patch, serial9.patch,
SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the
serialization specific configuration. Since this data is really internal to the specific serialization,
I think we should change it to be an opaque binary blob. This will simplify the interface
for defining specific serializations for different contexts (MAPREDUCE-1462). It will also
move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message