hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chaoyu Tang <ctang...@gmail.com>
Subject Re: Review Request 49952: HIVE-14205: Hive doesn't support union type with AVRO file format
Date Wed, 13 Jul 2016 01:11:42 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49952/#review141997
-----------------------------------------------------------



Thanks [~yshi] for patch. It looks good. But I have a couple of questions:
It seems to me that the union in existing code is only used to support Nullable type in Avro,
and has not been fully supported as a data type in general. This patch actually extends (or
adds) this type support. 
So with the patch, how can we be able to distinguish an Avro union between nullable and non-nullable,
for example, for following field schema, both might end with type uniontype<int, bigint>
{code}
      "fields":[
           {
             "name":"value",
             "type":[
                "null",
                 "int",
                 "long"
              ],
              "default":null
        ]
---
      "fields":[
           {
             "name":"value",
             "type":[
                 "int",
                 "long"
              ],
              "default": 0
        ]
{code}
Will there be any problem? Also could we add some qtests using Avro union data (with or without
null)?

- Chaoyu Tang


On July 12, 2016, 11:07 a.m., Yibing Shi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49952/
> -----------------------------------------------------------
> 
> (Updated July 12, 2016, 11:07 a.m.)
> 
> 
> Review request for hive and Chaoyu Tang.
> 
> 
> Bugs: HIVE-14205
>     https://issues.apache.org/jira/browse/HIVE-14205
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-14205: Hive doesn't support union type with AVRO file format
> 
> 
> Diffs
> -----
> 
>   serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java 6165138 
>   serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 08ee62b 
>   serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroDeserializer.java 986b803

>   serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 0013b78 
> 
> Diff: https://reviews.apache.org/r/49952/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Yibing Shi
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message