Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Message-ID: <17859369.84221291331060262.JavaMail.jira@thor>
Date: Thu, 2 Dec 2010 18:04:20 -0500 (EST)
From: "Doug Cutting (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-6685) Change the generic serialization
 framework API to use serialization-specific bytes instead of
 Map<String,String> for configuration
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966335#action_12966335 ] 

Doug Cutting commented on HADOOP-6685:
--------------------------------------

> I haven't heard of any use cases where the serializations would be nested.

True, serializations might not be directly nested, but they might be nested in other configuration data.  We're trying to devise a better configuration serialization pattern than Map<String,String> for use in MAPREDUCE-1183 and MAPREDUCE-1462 .  In these cases, jobs, serializations, mapper, inputformat, reducer and outputformat configurations are nested.

> I'd rather not bake JSON into the interface, although clearly a given serialization could choose to use JSON. I don't find JSON very user-friendly.

Java programmers should never need to touch the JSON (except perhaps when debugging) since Java programmers can use APIs to access configurations.  For Java programmers it should be equivalent to binary.  But it'd be nice if configurations of all sorts, daemons, jobs, serializations, inputformats, mappers, etc. were in a common format that it was possible for non-Java programs to see and potentially modify as they can currently with -D, text-editors, etc.  Currently we use Map<String,String> for this purpose, but that makes nesting awkward.  I see this issue primarily as a replacement for the reverted HADOOP-6420, a previous attempt to provide configuration nesting.  Is that a mis-characterization?

> Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: libthrift.jar, serial.patch, serial4.patch, serial6.patch, serial7.patch, SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the serialization specific configuration. Since this data is really internal to the specific serialization, I think we should change it to be an opaque binary blob. This will simplify the interface for defining specific serializations for different contexts (MAPREDUCE-1462). It will also move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.