hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6685) Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
Date Wed, 08 Dec 2010 19:53:10 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969454#action_12969454
] 

Doug Cutting commented on HADOOP-6685:
--------------------------------------

Allen> If I want to build a mass documentation signing system and provide the binary representation
of the CA cert as a configuration option to my serializer, why shouldn't I be able to do that?

You should be able to do that.  Support for binary and other typed configuration values would
be great to have.  I do not disagree with that.  What I'd like to not lose is a generic way
to access configuration values.  A standard configuration data model would permit that.

Owen> forcing all of the metadata into a format just to avoid quoting in one format is
extremely premature.

The lack of quoting is a feature, but not a motivating reason.  The motivating reason is to
continue to provide uniform access to configurations.  Folks today are able to set the value
of configuration parameters with -D.  Folks today are able to view & edit configurations.
 It is possible (if awkward) today to write a single piece of code that can retrieve any job
or daemon parameter from LDAP or SmartFrog.  If we move to an opaque system, with no uniform
data model that permits one to specify a path to a value and a (possibly typed) value then
one cannot perform such generic configuration tasks.  That would be a step backwards.

> Change the generic serialization framework API to use serialization-specific bytes instead
of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: serial.patch, serial4.patch, serial6.patch, serial7.patch, serial9.patch,
SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the
serialization specific configuration. Since this data is really internal to the specific serialization,
I think we should change it to be an opaque binary blob. This will simplify the interface
for defining specific serializations for different contexts (MAPREDUCE-1462). It will also
move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message