Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Message-ID: <13433556.130541289953396687.JavaMail.jira@thor>
Date: Tue, 16 Nov 2010 19:23:16 -0500 (EST)
From: "Tom White (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-6685) Change the generic serialization
 framework API to use serialization-specific bytes instead of
 Map<String,String> for configuration
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932727#action_12932727 ] 

Tom White commented on HADOOP-6685:
-----------------------------------

bq. Avro is already a dependency. Thrift is already a dependency for HDFS (see HDFS-1484).

There's a difference. Avro is used to implement an internal Hadoop format, and Thrift, which is used in the thriftfs contrib module is also an internal detail. So users don't care about the version numbers for these libraries. (Also, the latter is a contrib module, so users can elect not to include it.) The case I'm thinking about here is if a user has a Thrift file definition that uses a feature of a later version of Thrift than is included in Hadoop then they can't use it. If we make it a library then it becomes possible to update the library independently.

> Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: libthrift.jar, serial.patch, serial4.patch, serial6.patch, SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the serialization specific configuration. Since this data is really internal to the specific serialization, I think we should change it to be an opaque binary blob. This will simplify the interface for defining specific serializations for different contexts (MAPREDUCE-1462). It will also move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.