hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Barna Zsombor Klara <zsombor.kl...@cloudera.com>
Subject Re: Review Request 58501: HIVE-16469: Parquet timestamp table property is not always taken into account
Date Wed, 03 May 2017 12:59:34 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58501/
-----------------------------------------------------------

(Updated May 3, 2017, 12:59 p.m.)


Review request for hive, Sergio Pena and Zoltan Ivanfi.


Changes
-------

Updated based on comments.


Bugs: HIVE-16469
    https://issues.apache.org/jira/browse/HIVE-16469


Repository: hive-git


Description
-------

HIVE-16469: Parquet timestamp table property is not always taken into account


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 757b7fc0eaa39c956014aa446ab1b07fc4abf8d3

  ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java 13750cdc34711d22f2adf2f483a6773ad05fb8d2

  ql/src/java/org/apache/hadoop/hive/ql/exec/StatsNoJobTask.java 9c3a664b9aea2d6e050ffe2d7626127827dbc52a

  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java 1bd4db7805689ae1f91921ffbb5ff7da59f4bf60

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java f4fadbb61bf45f62945700284c0b050f0984b696

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ParquetRecordReaderBase.java 2954601ce5bb25905cdb29ca0ca4551c2ca12b95

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java 6413c5add6db2e8c9298285b15dba33ee74379a8

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetTableUtils.java b339cc4347eea143dca2f6d98f9aa7777afdc427

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/timestamp/NanoTimeUtils.java dbd6fb3d0bc8c753abf86e99b52377617f248b5a

  ql/src/test/org/apache/hadoop/hive/ql/io/parquet/AbstractTestParquetDirect.java c81499a91c84af3ba33f335506c1c44e7085f13d

  ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestParquetRowGroupFilter.java bf363f32a3ac0a4d790e2925d802c6e210adfb4b

  ql/src/test/org/apache/hadoop/hive/ql/io/parquet/VectorizedColumnReaderTestBase.java f2d79cf9d215e9a6e2a5e88cfc78378be860fd1f

  ql/src/test/org/apache/hadoop/hive/ql/io/parquet/timestamp/TestNanoTimeUtils.java 1e10dbf18742524982606f1e6c6d447d683b2dc3

  ql/src/test/queries/clientnegative/parquet_int96_alter_invalid_timezone.q PRE-CREATION 
  ql/src/test/queries/clientnegative/parquet_int96_create_invalid_timezone.q PRE-CREATION

  ql/src/test/queries/clientpositive/parquet_int96_timestamp.q 6eadd1b0a3313cbba7a798890b802baae302749e

  ql/src/test/results/clientnegative/parquet_int96_alter_invalid_timezone.q.out PRE-CREATION

  ql/src/test/results/clientnegative/parquet_int96_create_invalid_timezone.q.out PRE-CREATION

  ql/src/test/results/clientpositive/parquet_int96_timestamp.q.out b9a3664458a83f1856e4bc59eba5d56665df61cc

  ql/src/test/results/clientpositive/spark/parquet_int96_timestamp.q.out PRE-CREATION 


Diff: https://reviews.apache.org/r/58501/diff/4/

Changes: https://reviews.apache.org/r/58501/diff/3-4/


Testing
-------

Added qtests for the following cases:
- order by clause
- selfjoin
- calling UDFs with the timestamp values
- where clause with a constant cast as timestamp
- test for HoS
- implicit and explicit timestamp conversions in insert clause

Tested manually but no qtests:
- join between 3 tables all parquet but with different/no timezone property
- subselect in from/where clauses
- exists / union / no exists


Thanks,

Barna Zsombor Klara


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message