hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue" <b...@apache.org>
Subject Re: Review Request 32499: HIVE-10086: Hive throws error when accessing Parquet file schema using field name match
Date Thu, 26 Mar 2015 20:59:16 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32499/#review77949
-----------------------------------------------------------

Ship it!


Overall looks fine.


ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java
<https://reviews.apache.org/r/32499/#comment126310>

    Nit: This could be parameterized to avoid the cast a couple lines later. (Forgot to meniton
it in the first round).



ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java
<https://reviews.apache.org/r/32499/#comment126309>

    Why `new Type[0]` instead of `new Type[projectedFields.size()]`?


- Ryan Blue


On March 26, 2015, 1:51 p.m., Sergio Pena wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32499/
> -----------------------------------------------------------
> 
> (Updated March 26, 2015, 1:51 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-10086
>     https://issues.apache.org/jira/browse/HIVE-10086
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Attached is the patch that handles schema that do not match between Parquet and Hive.
> 
> The access to Parquet data is with name matching in this case. The table column may have
different schema order, but if the name matches the parquet column name, then the value is
retrieved.
> 
> Also, if the Hive schema has columns and struct elements that do not match with the Parquet
schema, then it will return NULL values instead.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/DataWritableRecordConverter.java
a43661eb54ba29692c07c264584b5aecf648ef99 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java
57ae7a9740d55b407cadfc8bc030593b29f90700 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java
a26199612cf338e336f210f29acb0398c536e1f9 
>   ql/src/test/queries/clientpositive/parquet_schema_evolution.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_table_with_subschema.q PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_schema_evolution.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_table_with_subschema.q.out PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/32499/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message