hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohit Sabharwal" <mo...@cloudera.com>
Subject Re: Review Request 24713: HIVE-7735 : Implement Char, Varchar in ParquetSerDe
Date Fri, 15 Aug 2014 20:28:58 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24713/
-----------------------------------------------------------

(Updated Aug. 15, 2014, 8:28 p.m.)


Review request for hive.


Changes
-------

Incorporated feedback.


Bugs: HIVE-7735
    https://issues.apache.org/jira/browse/HIVE-7735


Repository: hive-git


Description
-------

HIVE-7735 : Implement Char, Varchar in ParquetSerDe

- Since string, char and varchar are all represented as the same parquet
type (primitive type binary, original type utf8), this patch plumbs the
hive column types into ETypeConverter to distinguish between the three.

- Removes Decimal related dead code in ArrayWritableObjectInspector,
(decimal is supported in Parquet SerDe) 


Diffs (updated)
-----

  data/files/parquet_types.txt 9d81c3c3130cb94ae2bc308d511b0e24a60d4b8e 
  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ArrayWritableGroupConverter.java
582a5dfdaccaa25d46bfb515248eeb4bb84bedc5 
  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/DataWritableGroupConverter.java
0e310fbfb748d5409ff3c0d8cd8327bec9988ecf 
  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/DataWritableRecordConverter.java
7762afea4dda8cb4be4756eef43abec566ea8444 
  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java 67ce15187a33d58fda7ff5b629339bd89d0e5e54

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveGroupConverter.java 524a2937e39a4821a856c8e25b14633ade89ea49

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java 99901f0f57328db6fb2a260f7b7d76ded6f39558

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java d6be4bdfc1502cf79c184726d88eb0bd94fb2b02

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java
47bf69ce7cb6f474f9f48dd693a7915475a1d9cb 
  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java e3e327c7b657cdd397dd2b4dddf40187c65ce901

  ql/src/java/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java 0637d46f2f7162c8d617c761e817dcf396fc94fe

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java ba4ac690ccc361e65f12220997f300067bbd0d6c

  ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestHiveSchemaConverter.java b87cf7449679a9b6da997010056e388fb3de9945

  ql/src/test/queries/clientnegative/parquet_char.q 745a7867264e321c079d8146f60d14ae186bbc29

  ql/src/test/queries/clientnegative/parquet_varchar.q 55825f76dc240c54ef451ceec12adee23f12b36c

  ql/src/test/queries/clientpositive/parquet_types.q cb0dcfdf2d637854a84b165f8565fcb683617696

  ql/src/test/results/clientnegative/parquet_char.q.out 8c9a52c63416eaeaf99cb51b9f386f886483f29c

  ql/src/test/results/clientnegative/parquet_timestamp.q.out 00973b7e1f6360ce830a8baa4b959491ccc87a9b

  ql/src/test/results/clientnegative/parquet_varchar.q.out 90f6db25960825472270532811b8a17d9774d412

  ql/src/test/results/clientpositive/parquet_types.q.out 3acb0520bab023238e19b728ffedc3344c7f1a06

  serde/src/java/org/apache/hadoop/hive/serde2/Deserializer.java ade3b5f081eb71e5cf4e639aff8bff6447d68dfc

  serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.java e7f3f4837ab253a825a7210f56f595b2403e7385


Diff: https://reviews.apache.org/r/24713/diff/


Testing
-------

- Added char, varchar types in parquet_types q-test.
- Added unit test for char, varchar in TestHiveSchemaConverter
- Removed char, varchar negative q-test files.


Thanks,

Mohit Sabharwal


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message