avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianxiang Xiong <tianxiang.xi...@fundingcircle.com>
Subject Re: UUIDs in Avro Schemas
Date Fri, 06 Jan 2017 19:48:36 GMT
Are the listed logical types those that the Avro parser supports "out of
the box", i.e. if using the Avro 1.8.1 Java API, a logical date will be
automatically deserialized into a *java.util.Date* and vice-versa?

I see that the *LogicalTypes* class has a *uuid()* method, but there is no
corresponding *LogicalTypes.UUID* class. There's also no mention of a
logical UUID in the documentation
<https://avro.apache.org/docs/1.8.1/spec.html#Logical+Types>. What, then,
does the method do?

Is there a more detailed example of specifying a custom logical type? I
assume that after specifying a logical type in the JSON schema, we can
extend the parser so it can do the conversion automatically. There's an
example <https://github.com/mtth/avsc/wiki/Advanced-usage#logical-types> of
this for *avsc*, an Avro JavaScript library, but I haven't seen one for
Java.

On 23 November 2016 at 07:37, Sam Groth <sgroth@yahoo-inc.com> wrote:

> For UUIDs, I would probably use the fixed type, but a string would also
> work. See the specification for the different types that avro supports:
> http://avro.apache.org/docs/current/spec.html
>
>
> Sam
>
>
> On Tuesday, November 22, 2016 6:32 PM, Tianxiang Xiong <tianxiang.xiong@
> fundingcircle.com> wrote:
>
>
> I see, I'll close the ticket then if using logical types this way to
> specify UUID is idiomatic.
>
> On 22 November 2016 at 16:21, Yibing Shi <yshi@cloudera.com> wrote:
>
> Hi Tianxiang,
>
> I am sorry! I replied too quickly!
>
> I thought that you were complaining that you couldn't generate the UUID
> type with IDL compiler, but when I check the JIRA you have just created, I
> find that you actually want to know how to specify UUID type in JSON
> format, which has already been there. Please use below definition:
>
> {
>   "type": "string",
>   "logicalType": "uuid"
> }
>
> *Yibing Shi*
> *Customer Operations Engineer*
> <http://www.cloudera.com/>
>
> On Wed, Nov 23, 2016 at 10:43 AM, Tianxiang Xiong <tianxiang.xiong@
> fundingcircle.com <tianxiang.xiong@fundingcircle.com>> wrote:
>
> Thanks Yibing, I have created AVRO-1962
> <https://issues.apache.org/jira/browse/AVRO-1962>. Take a look and let me
> know if additional information/changes are needed to the ticket.
>
> On 22 November 2016 at 15:12, Yibing Shi <yshi@cloudera.com> wrote:
>
> Hi Tianxiang,
>
> Looks like that the UUID type hasn't been added to IDL definition yet.
>
> https://github.com/apache/avro /blob/master/lang/java/compile
> r/src/main/javacc/org/apache/a vro/compiler/idl/idl.jj#L214-L 244
> <https://github.com/apache/avro/blob/master/lang/java/compiler/src/main/javacc/org/apache/avro/compiler/idl/idl.jj#L214-L244>
>
> Please create a JIRA to track this.
>
> *Yibing Shi*
> *Customer Operations Engineer*
> <http://www.cloudera.com/>
>
> On Wed, Nov 23, 2016 at 4:58 AM, Tianxiang Xiong <tianxiang.xiong@fundingcircle
> .com <tianxiang.xiong@fundingcircle.com>> wrote:
>
> The AVRO-1554 <https://issues.apache.org/jir a/browse/AVRO-1554
> <https://issues.apache.org/jira/browse/AVRO-1554>> ticket
> seems to suggest that the issue of representing UUIDs is resolved with
> logical
> types <http://avro.apache.org/docs/1 .8.1/spec.html#Logical+Types
> <http://avro.apache.org/docs/1.8.1/spec.html#Logical+Types>> in Avro
> 1.8.1.
>
> The specification offers several examples of using logical types; decimals
> are represented as:
>
> {
>
>   "type": "bytes",
>   "logicalType": "decimal",
>   "precision": 4,
>   "scale": 2
> }
>
> However, there does not seem to be an example of UUID. The most
> popular answers <http://stackoverflow.com/a/19 705149/864684
> <http://stackoverflow.com/a/19705149/864684>> via
> Google/StackOverflow are a few years old and do not make use of
> logical types, and can therefore be misleading for newer versions of
> Avro
>
> Is there an idiomatic way of representing UUIDs in Avro Schemas?
>
> Thanks,
>
> --
> *Tianxiang Xiong *| Clojure Engineer
> *tianxiang.xiong@fundingcircle. com <tianxiang.xiong@fundingcircle.com>*
> 747 Front Street, Floor 4 | San Francisco, CA 94111
>
>
>
>
>
> --
> *Tianxiang Xiong *| Clojure Engineer
> *tianxiang.xiong@fundingcircle. com <tianxiang.xiong@fundingcircle.com>*
> 747 Front Street, Floor 4 | San Francisco, CA 94111
>
>
>
>
>
> --
> *Tianxiang Xiong *| Clojure Engineer
> *tianxiang.xiong@fundingcircle.com <tianxiang.xiong@fundingcircle.com>*
> 747 Front Street, Floor 4 | San Francisco, CA 94111
>
>
>


-- 

*Tianxiang Xiong *| Clojure Engineer

*tianxiang.xiong@fundingcircle.com <tianxiang.xiong@fundingcircle.com>*

747 Front Street, Floor 4 | San Francisco, CA 94111

Mime
View raw message