avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1402) Support for DECIMAL type
Date Tue, 08 Apr 2014 14:28:21 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963029#comment-13963029
] 

Tom White commented on AVRO-1402:
---------------------------------

There have been objections to the approach implemented here on the basis that there’s no
a clear use case for storing decimals with different scales in the same field. So here’s
another idea. Rather than adding new types into GenericData, provide helper code to make it
easy for applications to use decimals with some agreed upon schemas. To avoid the problems
with unique names we could go back to the logical type approach:

{code}
{"type":"bytes", "logicalType":"decimal", "scale":"2”} 
{code}

We’d then have code like

{code}
// write
Schema schema = DecimalHelper.schema(2);
BigDecimal decimal = new BigDecimal("12.45");
GenericDatumWriter<ByteBuffer> writer = new
    GenericDatumWriter<ByteBuffer>(schema, GenericData.get());
writer.write(DecimalHelper.toByteBuffer(decimal), encoder);

// read
int scale = DecimalHelper.getScale(schema);
GenericDatumReader<ByteBuffer> reader = new
  GenericDatumReader<ByteBuffer>(schema, schema, GenericData.get());
ByteBuffer buffer = reader.read(null, decoder);
BigDecimal readDecimal = DecimalHelper.fromByteBuffer(buffer, scale);  
{code}

If we did this we would revert the patch in this JIRA, since there’s no benefit in having
both approaches. Date/time could be implemented in the same way.

Also, the work on interchangeable memory-models (AVRO-1469) might provide a more flexible
approach for supporting more in-memory types (like BigDecimal) in the future.

> Support for DECIMAL type
> ------------------------
>
>                 Key: AVRO-1402
>                 URL: https://issues.apache.org/jira/browse/AVRO-1402
>             Project: Avro
>          Issue Type: New Feature
>    Affects Versions: 1.7.5
>            Reporter: Mariano Dominguez
>            Assignee: Tom White
>            Priority: Minor
>              Labels: Hive
>             Fix For: 1.7.7
>
>         Attachments: AVRO-1402.patch, AVRO-1402.patch, AVRO-1402.patch, AVRO-1402.patch,
UnixEpochRecordMapping.patch
>
>
> Currently, Avro does not seem to support a DECIMAL type or equivalent.
> http://avro.apache.org/docs/1.7.5/spec.html#schema_primitive
> Adding DECIMAL support would be particularly interesting when converting types from Avro
to Hive, since DECIMAL is already a supported data type in Hive (0.11.0).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message