hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6685) Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
Date Fri, 19 Nov 2010 19:30:19 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933926#action_12933926
] 

Owen O'Malley commented on HADOOP-6685:
---------------------------------------

{quote}
This sounds like a documentation issue, not a functional deficiency. This style is used consistently
throughout Hadoop. If we seek to replace Configuration that should perhaps be considered wholesale
rather than piecemeal.
{quote}

Not at all. Requiring the metadata to be string to string maps forced the implementation to
have a large number of related static functions. I should also point out that my proposed
framework can support text-based metadata, it just doesn't require it.

{quote}
I have not seen a large number of complaints from end users about this aspect of Hadoop.
{quote}

That demonstrates a lack of involvement with users, rather than the benefits of the current
system. Users often complain about the current configuration system. In particular, typos
that result in subtly wrong results are very user unfriendly. String to string maps are very
useful, but there are also a lot of benefits to having strongly typed interfaces.

Users are also unhappy with the static methods that update configurations for doing things
like setting the input and output directories.


> Change the generic serialization framework API to use serialization-specific bytes instead
of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: libthrift.jar, serial.patch, serial4.patch, serial6.patch, serial7.patch,
SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the
serialization specific configuration. Since this data is really internal to the specific serialization,
I think we should change it to be an opaque binary blob. This will simplify the interface
for defining specific serializations for different contexts (MAPREDUCE-1462). It will also
move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message