hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6120) Add support for Avro types in hadoop
Date Mon, 20 Jul 2009 19:38:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733331#action_12733331
] 

Doug Cutting commented on HADOOP-6120:
--------------------------------------

If, e.g., a class both implements Writable and also either implements AvroReflectSerializeable
or whose package is listed in the config, then which serialization is used is determined by
the order of the serializers in the config, which could change.  Similarly for the Serializeable
interface and JavaSerialization.  So its not safe to assume that the Class->Serialization
map is fixed, and we should really be storing at least the serialization's name in container
metadata, and probably also a version (e.g. serialVersionUID for JavaSerialization).  And
once we have a mechanism to support that, we can also store other metadata, like the schema,
so that we can read older versions and generic data.

+1 for removing JavaSerialization from the defaults.

+1 for a separate Jira on serialization metadata.  But it would be best to have such metadata
by the 0.21 freeze, in two weeks.


> Add support for Avro types in hadoop
> ------------------------------------
>
>                 Key: HADOOP-6120
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6120
>             Project: Hadoop Common
>          Issue Type: New Feature
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>         Attachments: 6120_v1.patch, 6120_v2.patch, 6120_v3.patch, 6120_v4.patch, 6120_v5.patch,
HADOOP-6120.patch
>
>
> Support to serialize and deserialize Avro types in Hadoop.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message