hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Na Yang" <ny...@maprtech.com>
Subject Re: Review Request 25176: HIVE-7870: Insert overwrite table query does not generate correct task plan [Spark Branch]
Date Fri, 29 Aug 2014 06:44:25 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25176/
-----------------------------------------------------------

(Updated Aug. 29, 2014, 6:44 a.m.)


Review request for hive, Brock Noland, Szehon Ho, and Xuefu Zhang.


Changes
-------

1. add .q.out for TestCliDriver test for all new spark .q tests
2. update existing .q.out files because of plan change


Bugs: HIVE-7870
    https://issues.apache.org/jira/browse/HIVE-7870


Repository: hive-git


Description
-------

HIVE-7870: Insert overwrite table query does not generate correct task plan [Spark Branch]

The cause of this problem is during spark/tez task generation, the union file sink operator
are cloned to two new filesink operator. The linkedfilesinkdesc info for those new filesink
operators are missing. In addition, the two new filesink operators also need to be linked
together.   


Diffs (updated)
-----

  itests/src/test/resources/testconfiguration.properties 6393671 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 9c808d4 
  ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkProcContext.java 5ddc16d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkUtils.java 379a39c 
  ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java 76fc290 
  ql/src/test/queries/clientpositive/union_remove_spark_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_10.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_11.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_15.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_16.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_17.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_18.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_19.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_2.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_20.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_21.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_24.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_25.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_3.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_4.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_5.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_6.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_7.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_8.q PRE-CREATION 
  ql/src/test/queries/clientpositive/union_remove_spark_9.q PRE-CREATION 
  ql/src/test/results/clientpositive/spark/sample8.q.out c7e333b 
  ql/src/test/results/clientpositive/spark/union10.q.out 20c681e 
  ql/src/test/results/clientpositive/spark/union18.q.out 3f37a0a 
  ql/src/test/results/clientpositive/spark/union19.q.out 6922fcd 
  ql/src/test/results/clientpositive/spark/union28.q.out 8bd5218 
  ql/src/test/results/clientpositive/spark/union29.q.out b9546ef 
  ql/src/test/results/clientpositive/spark/union3.q.out 3ae6536 
  ql/src/test/results/clientpositive/spark/union30.q.out 12717a1 
  ql/src/test/results/clientpositive/spark/union33.q.out b89757f 
  ql/src/test/results/clientpositive/spark/union4.q.out 6341cd9 
  ql/src/test/results/clientpositive/spark/union6.q.out 263d9f4 
  ql/src/test/results/clientpositive/spark/union_remove_spark_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_10.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_11.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_15.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_16.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_17.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_18.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_19.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_20.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_21.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_24.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_25.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_4.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_5.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_6.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_7.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_8.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/union_remove_spark_9.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_10.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_11.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_15.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_16.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_17.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_18.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_19.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_20.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_21.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_24.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_25.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_4.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_5.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_6.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_7.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_8.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_remove_spark_9.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/25176/diff/


Testing
-------


Thanks,

Na Yang


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message