hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6660) add wrapper for Avro data
Date Thu, 25 Mar 2010 21:27:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849906#action_12849906

Doug Cutting commented on HADOOP-6660:

We might have static methods to configure a job for use with Avro data e.g.:

public static void setMapOutputKeyGeneric(Configuration, Schema);
public static void setMapOutputKeySpecific(Configuration, Schema);
public static void setOutputKeyGeneric(Configuration, Schema);
public static void setOutputKeySpecific(Configuration, Schema);
public static void setMapOutputValueGeneric(Configuration, Schema);
public static void setMapOutputValueSpecific(Configuration, Schema);
public static void setOutputValueGeneric(Configuration, Schema);
public static void setOutputValueSpecific(Configuration, Schema);

These would set in the configuration:
 - the appropriate output class name key (key/value map/reduce)
 - the appropriate schema key (key/value map/reduce)
 - the appropriate specific/generic key (key/value map/reduce)
Thus, in total, there would be twelve keys.

The serialization implementation would then use these to construct an appropriate deserializer.

These methods would not depend on any MapReduce code, and should reside with the serialization
implementation, so I propose to add them here.

> add wrapper for Avro data
> -------------------------
>                 Key: HADOOP-6660
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6660
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: io
>            Reporter: Doug Cutting
>             Fix For: 0.22.0
> To permit passing Avro data through MapReduce we can add a wrapper class and serialization
for it.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message