hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergio Pena" <sergio.p...@cloudera.com>
Subject Re: Review Request 27404: HIVE-8359 Map containing null values are not correctly written in Parquet files
Date Sat, 15 Nov 2014 00:06:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27404/
-----------------------------------------------------------

(Updated Nov. 15, 2014, 12:06 a.m.)


Review request for hive, Ryan Blue and Brock Noland.


Changes
-------

- More code refactoring to make DataWritableWriter.java more readable.
- Remove backward compatibles unit tests as they won't be used.


Bugs: HIVE-8359
    https://issues.apache.org/jira/browse/HIVE-8359


Repository: hive-git


Description
-------

The patch changes the way DataWritableWriter class writes an array of elements to the Parquet
record. 
It wraps each array record into a new startGroup/endGroup block so that Parquet can detect
null values on those optional fields.


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java c7078efe27482df0a11dd68ac068da27dbcf51b3

  ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestDataWritableWriter.java PRE-CREATION

  ql/src/test/queries/clientpositive/parquet_map_null.q PRE-CREATION 
  ql/src/test/results/clientpositive/parquet_map_null.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/27404/diff/


Testing
-------


Thanks,

Sergio Pena


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message