hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6420) String-to-String Maps should be embeddable in Configuration
Date Mon, 14 Dec 2009 21:09:48 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790370#action_12790370

Doug Cutting commented on HADOOP-6420:

> Have you looked into using a Stringifier or something along those lines?

That could be fun to boostrap.  DefaultStringifier uses SerializationFactory which could use
DefaultStringifier to find the serializer's metadata.

Whatever we use here we want to be easy for non-Java applications to use.  The original reason
for using a Map<String,String> model for configuration was that it is a proven format
for inter-operable configuration (environment variables).  So, if we used something like Stringifier,
we'd need to convince ourselves that the serialization we used for this was available to all
applications that might want to configure map-valued properties like serialization parameters.

I don't much like filtering, but I also don't much like big blobby stuff in configs.

Note that we don't need to filter the entire configuration unless someone iterates over the
subset.  The returned map view could instead be a wrapper map implementation that dynamically
prefixes keys on access to the underlying configuration.  If iteration is not required then
we could simply have it throw UnsupportedOperationException.  This would address the efficiency
and thread-safety concerns.

If we did need to support iteration, then perhaps we could change Configuration to use treemap,
but that might open other cans of worms.

> String-to-String Maps should be embeddable in Configuration
> -----------------------------------------------------------
>                 Key: HADOOP-6420
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6420
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: HADOOP-6420.2.patch, HADOOP-6420.3.patch, HADOOP-6420.4.patch, HADOOP-6420.patch
> Per MAPREDUCE-1126, we need to be able to take a map of (key, value) pairs and embed
that inside a Configuration object.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message