hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohit Sabharwal" <mo...@cloudera.com>
Subject Review Request 29325: HIVE-9193 : Fix ordering differences due to Java 8 (Part 3)
Date Mon, 22 Dec 2014 21:46:15 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29325/
-----------------------------------------------------------

Review request for hive.


Bugs: HIVE-9193
    https://issues.apache.org/jira/browse/HIVE-9193


Repository: hive-git


Description
-------

HIVE-9193 : Fix ordering differences due to Java 8 (Part 3)

This patch fixes following q-test failures in Java 8:
(1) avro_* q-tests with map data type.
Select operator uses LazySimpleSerDe/StandMapObjectInspector, which is used all over for serializing
results back to the end user for queries with maps. To avoid adding the additional memory
cost associated with LinkedHashMap, we generate java version specific out files for these
tests.

(2) plan_json.q, join0.q, input4.q, authorization_explain.q
EXPLAIN FORMATTED outputs string representation of JSONObject, which internally is a HashMap.
Again, we generate java version specific out files for these tests.

(3) explain_dependency.q
EXPLAIN DEPENDENCY outputs string representation of ReadEntity.parents. Changed that to an
ordered map.

(4) list_bucket_dml_6.q
DESC FORMATTED calls StorageDescriptor.getSkewedInfo() HMS API, which returns a thrift (unordered)
map. Generate java version specific out file for this test.


Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/hooks/ReadEntity.java b4f8432970708439ed90e896c725e0fa4287edfd

  ql/src/test/queries/clientpositive/authorization_explain.q 22bc89440f4dad6d200f7a52281c4da3fef98670

  ql/src/test/queries/clientpositive/avro_date.q 996fa47a32ea84fdf70b0a1db08ea857dcdff02d

  ql/src/test/queries/clientpositive/avro_deserialize_map_null.q fa170db53fac59e7142744900ab7c98badeddfa1

  ql/src/test/queries/clientpositive/avro_nullable_fields.q f90ceb96f5db93495398f9e736dd3049e7bfeac4

  ql/src/test/queries/clientpositive/avro_timestamp.q 7c04ede0065bc32bec19fe509b19092fca861403

  ql/src/test/queries/clientpositive/input4.q 1186bbbbe6bbd4542005fbe4060928537e2892aa 
  ql/src/test/queries/clientpositive/join0.q feac6d85714cb1daa736cc29a144cc4123216a75 
  ql/src/test/queries/clientpositive/list_bucket_dml_6.q 344525524b7587430618d43259307c045cca5a5b

  ql/src/test/queries/clientpositive/plan_json.q 0243c995412b04c43c43ab27aeb737dfbb6bd5e5

  ql/src/test/results/clientpositive/authorization_explain.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/authorization_explain.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/authorization_explain.q.out 58290cdcb74e200b7bdcd0dc1401bc52da159f06

  ql/src/test/results/clientpositive/avro_date.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/avro_date.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/avro_date.q.out 276a53c9dce898b6c167238507cb7f46ba4ffa78

  ql/src/test/results/clientpositive/avro_deserialize_map_null.q.java1.7.out PRE-CREATION

  ql/src/test/results/clientpositive/avro_deserialize_map_null.q.java1.8.out PRE-CREATION

  ql/src/test/results/clientpositive/avro_deserialize_map_null.q.out de79d6228fb1ed9cd9bb32e115ed53217a69bdbe

  ql/src/test/results/clientpositive/avro_nullable_fields.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/avro_nullable_fields.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/avro_nullable_fields.q.out 5dbec57a1bbae378f4ce0f39f7ccfc38c39977c7

  ql/src/test/results/clientpositive/avro_timestamp.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/avro_timestamp.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/avro_timestamp.q.out 3faa9d4a98b93bf8583d49affec76fce1e6f5933

  ql/src/test/results/clientpositive/explain_dependency.q.out fa4dc59a981fc584ac1aac124fb1fc4f9ce663be

  ql/src/test/results/clientpositive/input4.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/input4.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/input4.q.out 4b817611c7b799570bbd53c71f969fae5983c606

  ql/src/test/results/clientpositive/join0.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/join0.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/join0.q.out 5278e88d76b21067e749cccbd2e8692be1236606

  ql/src/test/results/clientpositive/list_bucket_dml_6.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/list_bucket_dml_6.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/list_bucket_dml_6.q.out 9930a906036bb5c59e98ada8b3f3a4c4d1d7f443

  ql/src/test/results/clientpositive/plan_json.q.java1.7.out PRE-CREATION 
  ql/src/test/results/clientpositive/plan_json.q.java1.8.out PRE-CREATION 
  ql/src/test/results/clientpositive/plan_json.q.out 5dc714b21854ca76c0d7e4492d1b9df070980d0d


Diff: https://reviews.apache.org/r/29325/diff/


Testing
-------

Tests pass for both Java 7 and Java 8


Thanks,

Mohit Sabharwal


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message