avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1402) Support for DECIMAL primitive type
Date Fri, 21 Mar 2014 21:56:44 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13943615#comment-13943615

Doug Cutting commented on AVRO-1402:

> If an application isn't setting the read schema and a file is written with a changed
schema, then there isn't a strong compatibility guarantee. 

In this case, for Java applications that use BigDecimal, upgrading Avro alone will change
the schema.  So the writing application's schema may change without any other change to the
application.  That alone may be a compatibility problem.

BTW, we may also need to remove much of the stringable handling for BigDecimal in SpecificData.
 We don't want both isString() and isBytes() to be true for the same object.  However we do
still need to be able to read instances of the old "java-class" schema for BigDecimal, and
perhaps even continue to write these.  Ugh.

I wonder if we instead should use something like serializers, e.g.,:

interface TypeMapping<T> {
  boolean isInstance(Object o);
  Schema getSchema();
  void write(T instance, Schema schema, Encoder encoder) throws IOException;
  T read(Schema schema, Decoder decoder) throws IOException;
public class DecimalMapping implements TypeMapping<BigDecimal> {
  private int scale;
  public DecimalMapping(int scale) { this.scale = scale; }

To opt into using this, you'd do something like:
GenericData data = new GenericData();
data.addTypeMapping(new DecimalMapping(10));

An application would need to update both its write and read side to start using this.  That
way each time we add a feature like this we won't need to make an incompatible release.

> Support for DECIMAL primitive type
> ----------------------------------
>                 Key: AVRO-1402
>                 URL: https://issues.apache.org/jira/browse/AVRO-1402
>             Project: Avro
>          Issue Type: New Feature
>    Affects Versions: 1.7.5
>            Reporter: Mariano Dominguez
>            Priority: Minor
>              Labels: Hive
>         Attachments: AVRO-1402.patch
> Currently, Avro does not seem to support a DECIMAL type or equivalent.
> http://avro.apache.org/docs/1.7.5/spec.html#schema_primitive
> Adding DECIMAL support would be particularly interesting when converting types from Avro
to Hive, since DECIMAL is already a supported data type in Hive (0.11.0).

This message was sent by Atlassian JIRA

View raw message