hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohit Sabharwal" <mo...@cloudera.com>
Subject Review Request 29520: HIVE-9239 : Fix ordering differences due to Java 8 (Part 5)
Date Wed, 31 Dec 2014 23:14:05 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29520/
-----------------------------------------------------------

Review request for hive.


Bugs: HIVE-9239
    https://issues.apache.org/jira/browse/HIVE-9239


Repository: hive-git


Description
-------

HIVE-9239 : Fix ordering differences due to Java 8 (Part 5)

This patch fixes the following tests:

(1) TestCliDriver.testCliDriver_column_access_stats
Converted all maps used to populate ColumnAccessInfo.tableToColumnAccessMap to deterministic
order maps. These include ParseContext.topToTable and relevant maps in UpdateDeleteSemanticAnalyzer.

(2) TestCliDriver.testCliDriver_table_access_keys_stats
CheckTableAccessHook prints table access key information from TableAccessInfo by operator.
Made the relevant maps deterministic.

(3) TestCliDriver.testCliDriver_subquery_multiinsert
Tasks are printed in different order in ExplainTask in Java 8 vs Java 7. TaskCompiler, specifically
MapReduceCompiler should use deterministic map for GenMRProcContext.opTaskMap

(4) TestCliDriver: outer_join_ppr.q and subquery_notin_having.q
Predicate ExprNodeDesc has children in different order. This ordering looks to be coming from
antlr. Generated version specific output for these.

(5) TestCliDriver.testCliDriver_parquet_map_null
Avro API uses HashMap for deserialized map, which makes input to Parquet non-deterministic.
Generated version specific output.


Diffs
-----

  itests/util/src/main/java/org/apache/hadoop/hive/ql/hooks/CheckColumnAccessHook.java f6058e4b1e74a0d77d3d44d009fe52429e593ae8

  itests/util/src/main/java/org/apache/hadoop/hive/ql/hooks/CheckTableAccessHook.java 8e19fad85092321e1fb1dc3494f0c83aa4445580

  ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnAccessInfo.java 8416cff1001eb452a4d977163965b0f15e9115fe

  ql/src/java/org/apache/hadoop/hive/ql/parse/MapReduceCompiler.java 703c9d119685dabe7f2e6c4f73877172841b4a2f

  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java d9da15e65f7cae18939082e4c7709bfb7b60bf60

  ql/src/java/org/apache/hadoop/hive/ql/parse/TableAccessAnalyzer.java d847a16b4141aa876baeeab01ec4d4cff61e290d

  ql/src/java/org/apache/hadoop/hive/ql/parse/TableAccessInfo.java b69830dfb73075b96bc23f5760d327d4477e26b0

  ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java 65f5eaa6b7370455494b30111051339091180a88

  ql/src/test/queries/clientpositive/outer_join_ppr.q c32e02305930b4e278cf74f29d794f1eb337f97b

  ql/src/test/queries/clientpositive/parquet_map_null.q d31cb9908a1f6aaea4eebc6dd1b69c8bb7ec9cda

  ql/src/test/queries/clientpositive/subquery_notin_having.q a181d593cc3006067ce84628af06d437c46a0a68

  ql/src/test/results/clientpositive/column_access_stats.q.out 9330ff7341e106317d2ed880d8e557e7c3f474dd

  ql/src/test/results/clientpositive/outer_join_ppr.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/outer_join_ppr.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/outer_join_ppr.q.out 58369ea2b8c53f3695f0301fc09a34684434ddc3

  ql/src/test/results/clientpositive/parquet_map_null.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/parquet_map_null.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/parquet_map_null.q.out 5772ef2811adf1e2b50972956db7be0a26ac3a97

  ql/src/test/results/clientpositive/subquery_multiinsert.q.out 556e2891f197e8c4b0180f14597118543bb8d5dd

  ql/src/test/results/clientpositive/subquery_notin_having.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/subquery_notin_having.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/subquery_notin_having.q.out 2e84283f3936a11a0e7a76c8e2da909c242ce6a5

  ql/src/test/results/clientpositive/table_access_keys_stats.q.out a0449aa39f8fc7960fe42680781a9f5ba01d42b8


Diff: https://reviews.apache.org/r/29520/diff/


Testing
-------

Tested with both Java 7 and Java 8


Thanks,

Mohit Sabharwal


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message