Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 848C9200C5E for ; Sat, 8 Apr 2017 05:58:08 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 8309B160B97; Sat, 8 Apr 2017 03:58:08 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 41C14160BB0 for ; Sat, 8 Apr 2017 05:58:06 +0200 (CEST) Received: (qmail 18443 invoked by uid 500); 8 Apr 2017 03:58:04 -0000 Mailing-List: contact commits-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hive-dev@hive.apache.org Delivered-To: mailing list commits@hive.apache.org Received: (qmail 16247 invoked by uid 99); 8 Apr 2017 03:58:03 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 08 Apr 2017 03:58:03 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 2D254E962B; Sat, 8 Apr 2017 03:58:03 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: hashutosh@apache.org To: commits@hive.apache.org Date: Sat, 08 Apr 2017 03:58:15 -0000 Message-Id: <8bf36e5931774b97984fdfc0995e572d@git.apache.org> In-Reply-To: <37b3688dff3e41dfbea92086a9278b21@git.apache.org> References: <37b3688dff3e41dfbea92086a9278b21@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [14/14] hive git commit: HIVE-14564 : Column Pruning generates out of order columns in SelectOperator which cause ArrayIndexOutOfBoundsException. (Zhihai Xu via Ashutosh Chauhan) archived-at: Sat, 08 Apr 2017 03:58:08 -0000 HIVE-14564 : Column Pruning generates out of order columns in SelectOperator which cause ArrayIndexOutOfBoundsException. (Zhihai Xu via Ashutosh Chauhan) Signed-off-by: Ashutosh Chauhan Project: http://git-wip-us.apache.org/repos/asf/hive/repo Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/78d8b337 Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/78d8b337 Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/78d8b337 Branch: refs/heads/master Commit: 78d8b337778993a2e24abb18cfd70e0976bd93ad Parents: ccc96b4 Author: Zhihai Xu Authored: Wed Aug 17 17:41:00 2016 -0700 Committer: Ashutosh Chauhan Committed: Fri Apr 7 20:56:44 2017 -0700 ---------------------------------------------------------------------- .../ql/optimizer/ColumnPrunerProcFactory.java | 15 +- .../clientpositive/select_column_pruning.q | 4 + .../clientpositive/autoColumnStats_8.q.out | 4 +- .../clientpositive/columnstats_partlvl.q.out | 4 +- .../clientpositive/columnstats_partlvl_dp.q.out | 16 +- .../clientpositive/columnstats_tbllvl.q.out | 16 +- .../results/clientpositive/complex_alias.q.out | 8 +- .../clientpositive/constant_prop_3.q.out | 4 +- .../clientpositive/correlationoptimizer13.q.out | 8 +- .../results/clientpositive/decimal_udf.q.out | 12 +- .../display_colstats_tbllvl.q.out | 8 +- .../distinct_windowing_no_cbo.q.out | 4 +- .../results/clientpositive/druid_basic2.q.out | 8 +- .../clientpositive/dynamic_rdd_cache.q.out | 52 +- .../results/clientpositive/except_all.q.out | 16 +- .../test/results/clientpositive/groupby9.q.out | 8 +- .../clientpositive/groupby_join_pushdown.q.out | 26 +- .../clientpositive/groupby_position.q.out | 26 +- .../test/results/clientpositive/having2.q.out | 86 +- .../clientpositive/index_auto_update.q.out | 4 +- .../limit_pushdown_negative.q.out | 4 +- .../clientpositive/llap/bucket_groupby.q.out | 4 +- .../llap/column_table_stats.q.out | 12 +- .../llap/column_table_stats_orc.q.out | 8 +- .../llap/dynamic_partition_pruning_2.q.out | 72 +- .../clientpositive/llap/explainuser_1.q.out | 20 +- .../clientpositive/llap/explainuser_2.q.out | 382 +++--- .../clientpositive/llap/lateral_view.q.out | 4 +- .../clientpositive/llap/limit_pushdown.q.out | 4 +- .../clientpositive/llap/limit_pushdown3.q.out | 4 +- .../clientpositive/llap/llap_stats.q.out | 4 +- .../llap/multi_insert_lateral_view.q.out | 92 +- .../llap/offset_limit_ppd_optimizer.q.out | 4 +- .../test/results/clientpositive/llap/ptf.q.out | 12 +- .../clientpositive/llap/subquery_in.q.out | 76 +- .../clientpositive/llap/subquery_multi.q.out | 200 ++-- .../clientpositive/llap/subquery_notin.q.out | 236 ++-- .../clientpositive/llap/subquery_scalar.q.out | 208 ++-- .../clientpositive/llap/subquery_select.q.out | 100 +- .../llap/vector_decimal_aggregate.q.out | 12 +- .../llap/vector_decimal_round_2.q.out | 6 +- .../llap/vector_decimal_udf.q.out | 12 +- .../clientpositive/llap/vector_groupby_3.q.out | 6 +- .../llap/vector_groupby_reduce.q.out | 12 +- .../clientpositive/llap/vector_interval_2.q.out | 14 +- .../clientpositive/llap/vector_orderby_5.q.out | 6 +- .../clientpositive/llap/vectorization_0.q.out | 6 +- .../clientpositive/llap/vectorization_13.q.out | 12 +- .../clientpositive/llap/vectorization_15.q.out | 4 +- .../llap/vectorization_short_regress.q.out | 56 +- .../llap/vectorized_parquet.q.out | 4 +- .../llap/vectorized_parquet_types.q.out | 6 +- .../clientpositive/llap/vectorized_ptf.q.out | 12 +- .../clientpositive/llap/windowing_gby.q.out | 42 +- .../clientpositive/multi_insert_gby3.q.out | 4 +- .../clientpositive/multigroupby_singlemr.q.out | 4 +- .../clientpositive/nested_column_pruning.q.out | 8 +- ..._non_dictionary_encoding_vectorization.q.out | 4 +- .../parquet_types_vectorization.q.out | 4 +- .../results/clientpositive/perf/query1.q.out | 126 +- .../results/clientpositive/perf/query12.q.out | 2 +- .../results/clientpositive/perf/query15.q.out | 106 +- .../results/clientpositive/perf/query16.q.out | 48 +- .../results/clientpositive/perf/query17.q.out | 214 ++-- .../results/clientpositive/perf/query18.q.out | 2 +- .../results/clientpositive/perf/query19.q.out | 100 +- .../results/clientpositive/perf/query20.q.out | 70 +- .../results/clientpositive/perf/query22.q.out | 98 +- .../results/clientpositive/perf/query25.q.out | 212 ++-- .../results/clientpositive/perf/query26.q.out | 126 +- .../results/clientpositive/perf/query29.q.out | 212 ++-- .../results/clientpositive/perf/query3.q.out | 70 +- .../results/clientpositive/perf/query30.q.out | 212 ++-- .../results/clientpositive/perf/query31.q.out | 420 ++++--- .../results/clientpositive/perf/query38.q.out | 210 ++-- .../results/clientpositive/perf/query39.q.out | 196 ++- .../results/clientpositive/perf/query42.q.out | 70 +- .../results/clientpositive/perf/query46.q.out | 126 +- .../results/clientpositive/perf/query5.q.out | 302 +++-- .../results/clientpositive/perf/query51.q.out | 84 +- .../results/clientpositive/perf/query52.q.out | 70 +- .../results/clientpositive/perf/query54.q.out | 244 ++-- .../results/clientpositive/perf/query55.q.out | 70 +- .../results/clientpositive/perf/query58.q.out | 378 +++--- .../results/clientpositive/perf/query6.q.out | 10 +- .../results/clientpositive/perf/query64.q.out | 1116 +++++++++--------- .../results/clientpositive/perf/query65.q.out | 84 +- .../results/clientpositive/perf/query66.q.out | 2 +- .../results/clientpositive/perf/query67.q.out | 2 +- .../results/clientpositive/perf/query68.q.out | 126 +- .../results/clientpositive/perf/query69.q.out | 2 +- .../results/clientpositive/perf/query7.q.out | 126 +- .../results/clientpositive/perf/query70.q.out | 70 +- .../results/clientpositive/perf/query71.q.out | 202 ++-- .../results/clientpositive/perf/query72.q.out | 4 +- .../results/clientpositive/perf/query79.q.out | 98 +- .../results/clientpositive/perf/query8.q.out | 222 ++-- .../results/clientpositive/perf/query81.q.out | 216 ++-- .../results/clientpositive/perf/query83.q.out | 462 ++++---- .../results/clientpositive/perf/query85.q.out | 234 ++-- .../results/clientpositive/perf/query87.q.out | 210 ++-- .../results/clientpositive/perf/query89.q.out | 98 +- .../results/clientpositive/perf/query91.q.out | 178 ++- .../results/clientpositive/perf/query92.q.out | 42 +- .../results/clientpositive/perf/query97.q.out | 42 +- .../results/clientpositive/perf/query98.q.out | 70 +- .../test/results/clientpositive/ppd_gby.q.out | 48 +- .../test/results/clientpositive/ppd_gby2.q.out | 48 +- .../results/clientpositive/ppd_windowing1.q.out | 36 +- .../results/clientpositive/ptfgroupbyjoin.q.out | 40 +- .../reduce_deduplicate_extended2.q.out | 128 +- .../clientpositive/select_column_pruning.q.out | 141 +++ .../spark/dynamic_rdd_cache.q.out | 48 +- .../results/clientpositive/spark/groupby9.q.out | 8 +- .../clientpositive/spark/groupby_position.q.out | 22 +- .../clientpositive/spark/limit_pushdown.q.out | 4 +- .../spark/multi_insert_gby3.q.out | 4 +- .../spark/multi_insert_lateral_view.q.out | 92 +- .../spark/multigroupby_singlemr.q.out | 4 +- .../test/results/clientpositive/spark/ptf.q.out | 12 +- .../clientpositive/spark/subquery_in.q.out | 72 +- .../spark/vector_decimal_aggregate.q.out | 12 +- .../clientpositive/spark/vector_groupby_3.q.out | 6 +- .../clientpositive/spark/vector_orderby_5.q.out | 6 +- .../clientpositive/spark/vectorization_0.q.out | 6 +- .../clientpositive/spark/vectorization_13.q.out | 12 +- .../clientpositive/spark/vectorization_15.q.out | 4 +- .../spark/vectorization_short_regress.q.out | 56 +- .../clientpositive/spark/vectorized_ptf.q.out | 12 +- .../clientpositive/subquery_in_having.q.out | 32 +- .../clientpositive/subquery_notexists.q.out | 20 +- .../subquery_notexists_having.q.out | 24 +- .../clientpositive/subquery_notin_having.q.out | 4 +- .../subquery_unqualcolumnrefs.q.out | 4 +- .../temp_table_display_colstats_tbllvl.q.out | 8 +- .../tez/vectorization_limit.q.out | 52 +- .../udaf_binarysetfunctions.q.out | 20 +- .../vector_decimal_aggregate.q.out | 12 +- .../clientpositive/vector_decimal_round_2.q.out | 6 +- .../clientpositive/vector_groupby_3.q.out | 6 +- .../clientpositive/vector_groupby_reduce.q.out | 12 +- .../clientpositive/vector_orderby_5.q.out | 6 +- .../clientpositive/vectorization_13.q.out | 12 +- .../clientpositive/vectorization_15.q.out | 4 +- .../clientpositive/vectorization_limit.q.out | 50 +- .../vectorized_parquet_types.q.out | 6 +- .../results/clientpositive/windowing_gby2.q.out | 34 +- 147 files changed, 5114 insertions(+), 5412 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java ---------------------------------------------------------------------- diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java index 00ec03e..acafd73 100644 --- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java +++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java @@ -810,11 +810,18 @@ public final class ColumnPrunerProcFactory { ArrayList newOutputColumnNames = new ArrayList(); ArrayList rs_oldsignature = op.getSchema().getSignature(); ArrayList rs_newsignature = new ArrayList(); + // The pruning needs to preserve the order of columns in the input schema + Set colNames = new HashSet(); for (FieldNode col : cols) { - int index = originalOutputColumnNames.indexOf(col.getFieldName()); - newOutputColumnNames.add(col.getFieldName()); - newColList.add(originalColList.get(index)); - rs_newsignature.add(rs_oldsignature.get(index)); + colNames.add(col.getFieldName()); + } + for (int i = 0; i < originalOutputColumnNames.size(); i++) { + String colName = originalOutputColumnNames.get(i); + if (colNames.contains(colName)) { + newOutputColumnNames.add(colName); + newColList.add(originalColList.get(i)); + rs_newsignature.add(rs_oldsignature.get(i)); + } } op.getSchema().setSignature(rs_newsignature); conf.setColList(newColList); http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/queries/clientpositive/select_column_pruning.q ---------------------------------------------------------------------- diff --git a/ql/src/test/queries/clientpositive/select_column_pruning.q b/ql/src/test/queries/clientpositive/select_column_pruning.q new file mode 100644 index 0000000..207cd32 --- /dev/null +++ b/ql/src/test/queries/clientpositive/select_column_pruning.q @@ -0,0 +1,4 @@ +CREATE TABLE lv_table1( c1 STRING, c2 ARRAY, c3 INT, c4 CHAR(1), c5 STRING, c6 STRING, c7 STRING, c8 STRING, c9 STRING, c10 STRING, c11 STRING); +INSERT OVERWRITE TABLE lv_table1 SELECT 'abc ', array(1,2,3), 100, 't', 'test', 'test', 'test', 'test', 'test', 'test', 'test' FROM src; +EXPLAIN SELECT * FROM lv_table1 LATERAL VIEW explode(array(1,2,3)) myTable AS myCol WHERE c3 = 100 SORT BY c1 ASC, myCol ASC LIMIT 1; +SELECT * FROM lv_table1 LATERAL VIEW explode(array(1,2,3)) myTable AS myCol WHERE c3 = 100 SORT BY c1 ASC, myCol ASC LIMIT 1; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/autoColumnStats_8.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/autoColumnStats_8.q.out b/ql/src/test/results/clientpositive/autoColumnStats_8.q.out index 329a897..681d962 100644 --- a/ql/src/test/results/clientpositive/autoColumnStats_8.q.out +++ b/ql/src/test/results/clientpositive/autoColumnStats_8.q.out @@ -157,8 +157,8 @@ STAGE PLANS: GatherStats: true MultiFileSpray: false Select Operator - expressions: _col2 (type: string), _col0 (type: string), _col1 (type: string) - outputColumnNames: hr, key, value + expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string) + outputColumnNames: key, value, hr Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(key, 16), compute_stats(value, 16) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/columnstats_partlvl.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/columnstats_partlvl.q.out b/ql/src/test/results/clientpositive/columnstats_partlvl.q.out index c5bfafb..07d26e9 100644 --- a/ql/src/test/results/clientpositive/columnstats_partlvl.q.out +++ b/ql/src/test/results/clientpositive/columnstats_partlvl.q.out @@ -539,8 +539,8 @@ STAGE PLANS: alias: employee_part Statistics: Num rows: 2 Data size: 210 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: employeesalary (type: double), employeeid (type: int), employeename (type: string) - outputColumnNames: employeesalary, employeeid, employeename + expressions: employeeid (type: int), employeename (type: string), employeesalary (type: double) + outputColumnNames: employeeid, employeename, employeesalary Statistics: Num rows: 2 Data size: 210 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(employeeid, 16), compute_stats(employeename, 16) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out b/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out index 08fb42e..468d2e7 100644 --- a/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out +++ b/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out @@ -84,8 +84,8 @@ STAGE PLANS: alias: employee_part Statistics: Num rows: 1 Data size: 64 Basic stats: PARTIAL Column stats: NONE Select Operator - expressions: country (type: string), employeename (type: string), employeeid (type: int) - outputColumnNames: country, employeename, employeeid + expressions: employeeid (type: int), employeename (type: string), country (type: string) + outputColumnNames: employeeid, employeename, country Statistics: Num rows: 1 Data size: 64 Basic stats: PARTIAL Column stats: NONE Group By Operator aggregations: compute_stats(employeename, 16), compute_stats(employeeid, 16) @@ -162,8 +162,8 @@ STAGE PLANS: alias: employee_part Statistics: Num rows: 42 Data size: 169 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: country (type: string), employeeid (type: int) - outputColumnNames: country, employeeid + expressions: employeeid (type: int), country (type: string) + outputColumnNames: employeeid, country Statistics: Num rows: 42 Data size: 169 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(employeeid, 16) @@ -251,8 +251,8 @@ STAGE PLANS: alias: employee_part Statistics: Num rows: 116 Data size: 466 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: employeesalary (type: double), country (type: string), employeeid (type: int) - outputColumnNames: employeesalary, country, employeeid + expressions: employeeid (type: int), employeesalary (type: double), country (type: string) + outputColumnNames: employeeid, employeesalary, country Statistics: Num rows: 116 Data size: 466 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(employeeid, 16) @@ -339,8 +339,8 @@ STAGE PLANS: alias: employee_part Statistics: Num rows: 2 Data size: 466 Basic stats: PARTIAL Column stats: NONE Select Operator - expressions: employeesalary (type: double), country (type: string), employeeid (type: int), employeename (type: string) - outputColumnNames: employeesalary, country, employeeid, employeename + expressions: employeeid (type: int), employeename (type: string), employeesalary (type: double), country (type: string) + outputColumnNames: employeeid, employeename, employeesalary, country Statistics: Num rows: 2 Data size: 466 Basic stats: PARTIAL Column stats: NONE Group By Operator aggregations: compute_stats(employeeid, 16), compute_stats(employeename, 16) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out b/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out index e8aaf8a..462d4c1 100644 --- a/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out +++ b/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out @@ -56,8 +56,8 @@ STAGE PLANS: alias: uservisits_web_text_none Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float) - outputColumnNames: sourceip, avgtimeonsite, adrevenue + expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int) + outputColumnNames: sourceip, adrevenue, avgtimeonsite Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16) @@ -108,8 +108,8 @@ STAGE PLANS: Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE GatherStats: false Select Operator - expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float) - outputColumnNames: sourceip, avgtimeonsite, adrevenue + expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int) + outputColumnNames: sourceip, adrevenue, avgtimeonsite Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16) @@ -478,8 +478,8 @@ STAGE PLANS: alias: uservisits_in_dummy_db Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float) - outputColumnNames: sourceip, avgtimeonsite, adrevenue + expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int) + outputColumnNames: sourceip, adrevenue, avgtimeonsite Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16) @@ -530,8 +530,8 @@ STAGE PLANS: Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE GatherStats: false Select Operator - expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float) - outputColumnNames: sourceip, avgtimeonsite, adrevenue + expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int) + outputColumnNames: sourceip, adrevenue, avgtimeonsite Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/complex_alias.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/complex_alias.q.out b/ql/src/test/results/clientpositive/complex_alias.q.out index 361bc8d..5182153 100644 --- a/ql/src/test/results/clientpositive/complex_alias.q.out +++ b/ql/src/test/results/clientpositive/complex_alias.q.out @@ -156,8 +156,8 @@ STAGE PLANS: Union Statistics: Num rows: 2 Data size: 34 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: _col2 (type: int), _col1 (type: string) - outputColumnNames: _col2, _col1 + expressions: _col1 (type: string), _col2 (type: int) + outputColumnNames: _col1, _col2 Statistics: Num rows: 2 Data size: 34 Basic stats: COMPLETE Column stats: NONE Group By Operator keys: _col2 (type: int), _col1 (type: string) @@ -182,8 +182,8 @@ STAGE PLANS: Union Statistics: Num rows: 2 Data size: 34 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: _col2 (type: int), _col1 (type: string) - outputColumnNames: _col2, _col1 + expressions: _col1 (type: string), _col2 (type: int) + outputColumnNames: _col1, _col2 Statistics: Num rows: 2 Data size: 34 Basic stats: COMPLETE Column stats: NONE Group By Operator keys: _col2 (type: int), _col1 (type: string) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/constant_prop_3.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/constant_prop_3.q.out b/ql/src/test/results/clientpositive/constant_prop_3.q.out index ecf6e57..7296131 100644 --- a/ql/src/test/results/clientpositive/constant_prop_3.q.out +++ b/ql/src/test/results/clientpositive/constant_prop_3.q.out @@ -272,8 +272,8 @@ STAGE PLANS: predicate: ((_col6 = 0) or (_col9 is null and _col1 is not null and (_col7 >= _col6))) (type: boolean) Statistics: Num rows: 1 Data size: 18 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: _col3 (type: string), _col4 (type: string), _col5 (type: int), _col1 (type: int) - outputColumnNames: _col3, _col4, _col5, _col1 + expressions: _col1 (type: int), _col3 (type: string), _col4 (type: string), _col5 (type: int) + outputColumnNames: _col1, _col3, _col4, _col5 Statistics: Num rows: 1 Data size: 18 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count(DISTINCT _col1) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/correlationoptimizer13.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/correlationoptimizer13.q.out b/ql/src/test/results/clientpositive/correlationoptimizer13.q.out index 0bd27b4..21219ca 100644 --- a/ql/src/test/results/clientpositive/correlationoptimizer13.q.out +++ b/ql/src/test/results/clientpositive/correlationoptimizer13.q.out @@ -52,8 +52,8 @@ STAGE PLANS: predicate: ((c1 < 120) and c3 is not null) (type: boolean) Statistics: Num rows: 342 Data size: 7639 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: c1 (type: int), c3 (type: string) - outputColumnNames: _col1, _col0 + expressions: c3 (type: string), c1 (type: int) + outputColumnNames: _col0, _col1 Statistics: Num rows: 342 Data size: 7639 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count(1) @@ -74,8 +74,8 @@ STAGE PLANS: predicate: ((c2 > 100) and (c1 < 120) and c3 is not null) (type: boolean) Statistics: Num rows: 114 Data size: 2546 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: c1 (type: int), c3 (type: string) - outputColumnNames: _col1, _col0 + expressions: c3 (type: string), c1 (type: int) + outputColumnNames: _col0, _col1 Statistics: Num rows: 114 Data size: 2546 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count(1) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/decimal_udf.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/decimal_udf.q.out b/ql/src/test/results/clientpositive/decimal_udf.q.out index 6e72f77..515fc7d 100644 --- a/ql/src/test/results/clientpositive/decimal_udf.q.out +++ b/ql/src/test/results/clientpositive/decimal_udf.q.out @@ -1278,8 +1278,8 @@ STAGE PLANS: alias: decimal_udf Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: value (type: int), key (type: decimal(20,10)) - outputColumnNames: value, key + expressions: key (type: decimal(20,10)), value (type: int) + outputColumnNames: key, value Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: sum(key), count(key), avg(key) @@ -1849,8 +1849,8 @@ STAGE PLANS: alias: decimal_udf Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: value (type: int), key (type: decimal(20,10)) - outputColumnNames: value, key + expressions: key (type: decimal(20,10)), value (type: int) + outputColumnNames: key, value Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: stddev(key), variance(key) @@ -1926,8 +1926,8 @@ STAGE PLANS: alias: decimal_udf Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: value (type: int), key (type: decimal(20,10)) - outputColumnNames: value, key + expressions: key (type: decimal(20,10)), value (type: int) + outputColumnNames: key, value Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: stddev_samp(key), var_samp(key) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out b/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out index f7d1400..7b49e4a 100644 --- a/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out +++ b/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out @@ -72,8 +72,8 @@ STAGE PLANS: alias: uservisits_web_text_none Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float) - outputColumnNames: sourceip, avgtimeonsite, adrevenue + expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int) + outputColumnNames: sourceip, adrevenue, avgtimeonsite Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16) @@ -124,8 +124,8 @@ STAGE PLANS: Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE GatherStats: false Select Operator - expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float) - outputColumnNames: sourceip, avgtimeonsite, adrevenue + expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int) + outputColumnNames: sourceip, adrevenue, avgtimeonsite Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out b/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out index 6be1467..aac939f 100644 --- a/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out +++ b/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out @@ -684,8 +684,8 @@ STAGE PLANS: isPivotResult: true Statistics: Num rows: 127193 Data size: 1017544 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: _col1 (type: smallint), rank_window_0 (type: int) - outputColumnNames: _col1, rank_window_0 + expressions: rank_window_0 (type: int), _col1 (type: smallint) + outputColumnNames: rank_window_0, _col1 Statistics: Num rows: 127193 Data size: 1017544 Basic stats: COMPLETE Column stats: NONE Group By Operator keys: _col1 (type: smallint), rank_window_0 (type: int) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/druid_basic2.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/druid_basic2.q.out b/ql/src/test/results/clientpositive/druid_basic2.q.out index bc9410b..495ebbe 100644 --- a/ql/src/test/results/clientpositive/druid_basic2.q.out +++ b/ql/src/test/results/clientpositive/druid_basic2.q.out @@ -557,8 +557,8 @@ STAGE PLANS: Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE GatherStats: false Select Operator - expressions: robot (type: string), language (type: string), __time (type: timestamp), added (type: float), delta (type: float) - outputColumnNames: robot, language, __time, added, delta + expressions: __time (type: timestamp), robot (type: string), language (type: string), added (type: float), delta (type: float) + outputColumnNames: __time, robot, language, added, delta Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Group By Operator aggregations: max(added), sum(delta) @@ -771,8 +771,8 @@ STAGE PLANS: Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE GatherStats: false Select Operator - expressions: robot (type: string), language (type: string), __time (type: timestamp), added (type: float), delta (type: float) - outputColumnNames: robot, language, __time, added, delta + expressions: __time (type: timestamp), robot (type: string), language (type: string), added (type: float), delta (type: float) + outputColumnNames: __time, robot, language, added, delta Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Group By Operator aggregations: max(added), sum(delta) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out b/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out index cf31ec1..2abb819 100644 --- a/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out +++ b/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out @@ -1033,22 +1033,18 @@ STAGE PLANS: 1 _col0 (type: int) outputColumnNames: _col2, _col4, _col5, _col6 Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE - Select Operator - expressions: _col5 (type: int), _col4 (type: int), _col6 (type: string), _col2 (type: int) - outputColumnNames: _col5, _col4, _col6, _col2 + Group By Operator + aggregations: stddev_samp(_col2), avg(_col2) + keys: _col5 (type: int), _col4 (type: int), _col6 (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3, _col4 Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE - Group By Operator - aggregations: stddev_samp(_col2), avg(_col2) - keys: _col5 (type: int), _col4 (type: int), _col6 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2, _col3, _col4 - Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE - File Output Operator - compressed: true - table: - input format: org.apache.hadoop.mapred.SequenceFileInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat - serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe + File Output Operator + compressed: true + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe Stage: Stage-4 Map Reduce @@ -1270,22 +1266,18 @@ STAGE PLANS: 1 _col0 (type: int) outputColumnNames: _col2, _col4, _col5, _col6 Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE - Select Operator - expressions: _col5 (type: int), _col4 (type: int), _col6 (type: string), _col2 (type: int) - outputColumnNames: _col5, _col4, _col6, _col2 + Group By Operator + aggregations: stddev_samp(_col2), avg(_col2) + keys: _col5 (type: int), _col4 (type: int), _col6 (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3, _col4 Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE - Group By Operator - aggregations: stddev_samp(_col2), avg(_col2) - keys: _col5 (type: int), _col4 (type: int), _col6 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2, _col3, _col4 - Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE - File Output Operator - compressed: true - table: - input format: org.apache.hadoop.mapred.SequenceFileInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat - serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe + File Output Operator + compressed: true + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe Stage: Stage-13 Map Reduce http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/except_all.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/except_all.q.out b/ql/src/test/results/clientpositive/except_all.q.out index 5ab3c64..61c1967 100644 --- a/ql/src/test/results/clientpositive/except_all.q.out +++ b/ql/src/test/results/clientpositive/except_all.q.out @@ -260,8 +260,8 @@ STAGE PLANS: Union Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: _col0 (type: string), _col1 (type: string), (_col2 * _col3) (type: bigint), _col3 (type: bigint) - outputColumnNames: _col0, _col1, _col4, _col3 + expressions: _col0 (type: string), _col1 (type: string), _col3 (type: bigint), (_col2 * _col3) (type: bigint) + outputColumnNames: _col0, _col1, _col3, _col4 Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: sum(_col4), sum(_col3) @@ -279,8 +279,8 @@ STAGE PLANS: Union Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: _col0 (type: string), _col1 (type: string), (_col2 * _col3) (type: bigint), _col3 (type: bigint) - outputColumnNames: _col0, _col1, _col4, _col3 + expressions: _col0 (type: string), _col1 (type: string), _col3 (type: bigint), (_col2 * _col3) (type: bigint) + outputColumnNames: _col0, _col1, _col3, _col4 Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: sum(_col4), sum(_col3) @@ -437,8 +437,8 @@ STAGE PLANS: Union Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: _col0 (type: string), _col1 (type: string), (_col2 * _col3) (type: bigint), _col3 (type: bigint) - outputColumnNames: _col0, _col1, _col4, _col3 + expressions: _col0 (type: string), _col1 (type: string), _col3 (type: bigint), (_col2 * _col3) (type: bigint) + outputColumnNames: _col0, _col1, _col3, _col4 Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: sum(_col4), sum(_col3) @@ -456,8 +456,8 @@ STAGE PLANS: Union Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: _col0 (type: string), _col1 (type: string), (_col2 * _col3) (type: bigint), _col3 (type: bigint) - outputColumnNames: _col0, _col1, _col4, _col3 + expressions: _col0 (type: string), _col1 (type: string), _col3 (type: bigint), (_col2 * _col3) (type: bigint) + outputColumnNames: _col0, _col1, _col3, _col4 Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: sum(_col4), sum(_col3) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/groupby9.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/groupby9.q.out b/ql/src/test/results/clientpositive/groupby9.q.out index f4f1163..15ea185 100644 --- a/ql/src/test/results/clientpositive/groupby9.q.out +++ b/ql/src/test/results/clientpositive/groupby9.q.out @@ -839,8 +839,8 @@ STAGE PLANS: Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: value (type: string), key (type: string) - outputColumnNames: value, key + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count(DISTINCT substr(value, 5)) @@ -3193,8 +3193,8 @@ STAGE PLANS: Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: value (type: string), key (type: string) - outputColumnNames: value, key + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count(DISTINCT substr(value, 5)) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out b/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out index 70a6b0e..95d499a 100644 --- a/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out +++ b/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out @@ -588,22 +588,18 @@ STAGE PLANS: 1 _col1 (type: int) outputColumnNames: _col0, _col2, _col3 Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: _col0 (type: tinyint), _col3 (type: tinyint), _col2 (type: bigint) - outputColumnNames: _col0, _col3, _col2 + Group By Operator + aggregations: sum(_col2) + keys: _col0 (type: tinyint), _col3 (type: tinyint) + mode: hash + outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: sum(_col2) - keys: _col0 (type: tinyint), _col3 (type: tinyint) - mode: hash - outputColumnNames: _col0, _col1, _col2 - Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - table: - input format: org.apache.hadoop.mapred.SequenceFileInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat - serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe + File Output Operator + compressed: false + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe Stage: Stage-2 Map Reduce http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/groupby_position.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/groupby_position.q.out b/ql/src/test/results/clientpositive/groupby_position.q.out index ff27253..3ded69c 100644 --- a/ql/src/test/results/clientpositive/groupby_position.q.out +++ b/ql/src/test/results/clientpositive/groupby_position.q.out @@ -246,22 +246,18 @@ STAGE PLANS: Filter Operator predicate: (key < 20) (type: boolean) Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: value (type: string), key (type: string) - outputColumnNames: value, key + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: value (type: string), key (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3 Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT substr(value, 5)) - keys: value (type: string), key (type: string), substr(value, 5) (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2, _col3 - Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - table: - input format: org.apache.hadoop.mapred.SequenceFileInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat - serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe + File Output Operator + compressed: false + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe Reduce Operator Tree: Group By Operator aggregations: count(DISTINCT KEY._col1:0._col0) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/having2.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/having2.q.out b/ql/src/test/results/clientpositive/having2.q.out index 59151fa..83d367f 100644 --- a/ql/src/test/results/clientpositive/having2.q.out +++ b/ql/src/test/results/clientpositive/having2.q.out @@ -132,8 +132,8 @@ STAGE PLANS: alias: s1 Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Select Operator - expressions: customer_name (type: string), customer_balance (type: double), order_quantity (type: double), discount (type: double) - outputColumnNames: customer_name, customer_balance, order_quantity, discount + expressions: order_quantity (type: double), discount (type: double), customer_name (type: string), customer_balance (type: double) + outputColumnNames: order_quantity, discount, customer_name, customer_balance Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Group By Operator aggregations: sum(customer_balance), sum(order_quantity), count(discount) @@ -199,8 +199,8 @@ STAGE PLANS: alias: s1 Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Select Operator - expressions: customer_name (type: string), customer_balance (type: double), order_quantity (type: double), discount (type: double) - outputColumnNames: customer_name, customer_balance, order_quantity, discount + expressions: order_quantity (type: double), discount (type: double), customer_name (type: string), customer_balance (type: double) + outputColumnNames: order_quantity, discount, customer_name, customer_balance Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE Group By Operator aggregations: sum(customer_balance), sum(order_quantity), count(discount) @@ -310,22 +310,18 @@ STAGE PLANS: 1 _col0 (type: string) outputColumnNames: _col0, _col1, _col2, _col4 Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: _col1 (type: string), _col2 (type: double), _col0 (type: double), _col4 (type: string) - outputColumnNames: _col1, _col2, _col0, _col4 + Group By Operator + aggregations: sum(_col2), avg(_col0), count(_col4) + keys: _col1 (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3 Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: sum(_col2), avg(_col0), count(_col4) - keys: _col1 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2, _col3 - Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - table: - input format: org.apache.hadoop.mapred.SequenceFileInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat - serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe + File Output Operator + compressed: false + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe Stage: Stage-2 Map Reduce @@ -433,22 +429,18 @@ STAGE PLANS: 1 _col0 (type: string) outputColumnNames: _col0, _col1, _col2, _col4 Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: _col1 (type: string), _col2 (type: double), _col0 (type: double), _col4 (type: string) - outputColumnNames: _col1, _col2, _col0, _col4 + Group By Operator + aggregations: sum(_col2), avg(_col0), count(_col4) + keys: _col1 (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3 Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: sum(_col2), avg(_col0), count(_col4) - keys: _col1 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2, _col3 - Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - table: - input format: org.apache.hadoop.mapred.SequenceFileInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat - serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe + File Output Operator + compressed: false + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe Stage: Stage-2 Map Reduce @@ -560,22 +552,18 @@ STAGE PLANS: 1 _col0 (type: string) outputColumnNames: _col0, _col1, _col2, _col4 Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: _col1 (type: string), _col2 (type: double), _col0 (type: double), _col4 (type: string) - outputColumnNames: _col1, _col2, _col0, _col4 + Group By Operator + aggregations: sum(_col2), avg(_col0), count(_col4) + keys: _col1 (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3 Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: sum(_col2), avg(_col0), count(_col4) - keys: _col1 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2, _col3 - Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - table: - input format: org.apache.hadoop.mapred.SequenceFileInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat - serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe + File Output Operator + compressed: false + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe Stage: Stage-2 Map Reduce http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/index_auto_update.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/index_auto_update.q.out b/ql/src/test/results/clientpositive/index_auto_update.q.out index d3d2ca3..e7bc069 100644 --- a/ql/src/test/results/clientpositive/index_auto_update.q.out +++ b/ql/src/test/results/clientpositive/index_auto_update.q.out @@ -98,8 +98,8 @@ STAGE PLANS: TableScan alias: temp Select Operator - expressions: key (type: string), INPUT__FILE__NAME (type: string), BLOCK__OFFSET__INSIDE__FILE (type: bigint) - outputColumnNames: key, INPUT__FILE__NAME, BLOCK__OFFSET__INSIDE__FILE + expressions: key (type: string), BLOCK__OFFSET__INSIDE__FILE (type: bigint), INPUT__FILE__NAME (type: string) + outputColumnNames: key, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME Group By Operator aggregations: collect_set(BLOCK__OFFSET__INSIDE__FILE) keys: key (type: string), INPUT__FILE__NAME (type: string) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out b/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out index a92656b..35e6827 100644 --- a/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out +++ b/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out @@ -84,8 +84,8 @@ STAGE PLANS: alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: value (type: string), key (type: string) - outputColumnNames: value, key + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: sum(key) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out b/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out index cb7623e..4ac5ac9 100644 --- a/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out +++ b/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out @@ -1569,8 +1569,8 @@ STAGE PLANS: alias: clustergroupby Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: key (type: string), value (type: string) - outputColumnNames: _col1, _col0 + expressions: value (type: string), key (type: string) + outputColumnNames: _col0, _col1 Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count(1) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/column_table_stats.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/column_table_stats.q.out b/ql/src/test/results/clientpositive/llap/column_table_stats.q.out index e23d882..fb04ee8 100644 --- a/ql/src/test/results/clientpositive/llap/column_table_stats.q.out +++ b/ql/src/test/results/clientpositive/llap/column_table_stats.q.out @@ -336,8 +336,8 @@ STAGE PLANS: Statistics Aggregation Key Prefix: default.spart/ GatherStats: true Select Operator - expressions: ds (type: string), hr (type: string), key (type: string), value (type: string) - outputColumnNames: ds, hr, key, value + expressions: key (type: string), value (type: string), ds (type: string), hr (type: string) + outputColumnNames: key, value, ds, hr Statistics: Num rows: 58 Data size: 32968 Basic stats: COMPLETE Column stats: PARTIAL Group By Operator aggregations: compute_stats(key, 16), compute_stats(value, 16) @@ -753,8 +753,8 @@ STAGE PLANS: Statistics Aggregation Key Prefix: default.spart/ GatherStats: true Select Operator - expressions: ds (type: string), hr (type: string), key (type: string), value (type: string) - outputColumnNames: ds, hr, key, value + expressions: key (type: string), value (type: string), ds (type: string), hr (type: string) + outputColumnNames: key, value, ds, hr Statistics: Num rows: 58 Data size: 32968 Basic stats: COMPLETE Column stats: PARTIAL Group By Operator aggregations: compute_stats(key, 16), compute_stats(value, 16) @@ -1170,8 +1170,8 @@ STAGE PLANS: Statistics Aggregation Key Prefix: default.spart/ GatherStats: true Select Operator - expressions: ds (type: string), key (type: string), value (type: string) - outputColumnNames: ds, key, value + expressions: key (type: string), value (type: string), ds (type: string) + outputColumnNames: key, value, ds Statistics: Num rows: 29 Data size: 11148 Basic stats: COMPLETE Column stats: PARTIAL Group By Operator aggregations: compute_stats(key, 16), compute_stats(value, 16) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out b/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out index 5f35e08..d55cf30 100644 --- a/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out +++ b/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out @@ -333,8 +333,8 @@ STAGE PLANS: Statistics Aggregation Key Prefix: default.spart/ GatherStats: true Select Operator - expressions: ds (type: string), hr (type: string), key (type: string), value (type: string) - outputColumnNames: ds, hr, key, value + expressions: key (type: string), value (type: string), ds (type: string), hr (type: string) + outputColumnNames: key, value, ds, hr Statistics: Num rows: 2 Data size: 1076 Basic stats: COMPLETE Column stats: PARTIAL Group By Operator aggregations: compute_stats(key, 16), compute_stats(value, 16) @@ -738,8 +738,8 @@ STAGE PLANS: Statistics Aggregation Key Prefix: default.spart/ GatherStats: true Select Operator - expressions: ds (type: string), key (type: string), value (type: string) - outputColumnNames: ds, key, value + expressions: key (type: string), value (type: string), ds (type: string) + outputColumnNames: key, value, ds Statistics: Num rows: 1 Data size: 354 Basic stats: COMPLETE Column stats: PARTIAL Group By Operator aggregations: compute_stats(key, 16), compute_stats(value, 16) http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out index 9dab93d..e78429f 100644 --- a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out +++ b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out @@ -181,22 +181,18 @@ STAGE PLANS: input vertices: 1 Map 4 Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: _col3 (type: string), _col0 (type: decimal(10,0)) - outputColumnNames: _col3, _col0 + Group By Operator + aggregations: count(), sum(_col0) + keys: _col3 (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(), sum(_col0) - keys: _col3 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0)) + value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0)) Execution mode: llap LLAP IO: no inputs Map 4 @@ -354,22 +350,18 @@ STAGE PLANS: input vertices: 1 Map 4 Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: _col3 (type: string), _col0 (type: decimal(10,0)) - outputColumnNames: _col3, _col0 + Group By Operator + aggregations: count(), sum(_col0) + keys: _col3 (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(), sum(_col0) - keys: _col3 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0)) + value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0)) Execution mode: llap LLAP IO: no inputs Map 4 @@ -727,22 +719,18 @@ STAGE PLANS: input vertices: 1 Map 4 Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: _col3 (type: string), _col0 (type: decimal(10,0)) - outputColumnNames: _col3, _col0 + Group By Operator + aggregations: count(), sum(_col0) + keys: _col3 (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(), sum(_col0) - keys: _col3 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0)) + value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0)) Execution mode: llap LLAP IO: no inputs Map 4 http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/explainuser_1.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out index 790efeb..a69f2ad 100644 --- a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out +++ b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out @@ -471,7 +471,7 @@ Stage-0 Group By Operator [GBY_32] (rows=1 width=20) Output:["_col0","_col1","_col2"],aggregations:["count()"],keys:_col6, _col2 Select Operator [SEL_31] (rows=1 width=16) - Output:["_col6","_col2"] + Output:["_col2","_col6"] Filter Operator [FIL_30] (rows=1 width=16) predicate:(((_col1 > 0) or (_col6 >= 0)) and ((_col6 >= 1) or (_col2 >= 1)) and ((UDFToLong(_col6) + _col2) >= 0)) Merge Join Operator [MERGEJOIN_48] (rows=3 width=16) @@ -642,7 +642,7 @@ Stage-0 Group By Operator [GBY_31] (rows=1 width=20) Output:["_col0","_col1","_col2"],aggregations:["count()"],keys:_col6, _col2 Select Operator [SEL_30] (rows=1 width=20) - Output:["_col6","_col2"] + Output:["_col2","_col6"] Filter Operator [FIL_29] (rows=1 width=20) predicate:(((_col1 + _col4) >= 0) and ((_col1 > 0) or (_col6 >= 0)) and ((_col6 >= 1) or (_col2 >= 1)) and ((UDFToLong(_col6) + _col2) >= 0)) Merge Join Operator [MERGEJOIN_42] (rows=4 width=20) @@ -1841,12 +1841,10 @@ Stage-0 PartitionCols:_col0 Group By Operator [GBY_5] (rows=41 width=178) Output:["_col0","_col1"],keys:value, key - Select Operator [SEL_4] (rows=83 width=178) - Output:["value","key"] - Filter Operator [FIL_21] (rows=83 width=178) - predicate:((value = value) and (value > 'val_2')) - TableScan [TS_2] (rows=500 width=178) - default@src_cbo,a,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"] + Filter Operator [FIL_21] (rows=83 width=178) + predicate:((value = value) and (value > 'val_2')) + TableScan [TS_2] (rows=500 width=178) + default@src_cbo,a,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"] PREHOOK: query: explain select * from src_cbo b @@ -2424,7 +2422,7 @@ Stage-0 Group By Operator [GBY_5] (rows=1 width=114) Output:["_col0","_col1","_col2"],aggregations:["count()","count(p_name)"],keys:p_mfgr Select Operator [SEL_4] (rows=4 width=223) - Output:["p_mfgr","p_name"] + Output:["p_name","p_mfgr"] Filter Operator [FIL_29] (rows=4 width=223) predicate:((p_size < 10) and (p_mfgr = p_mfgr)) TableScan [TS_2] (rows=26 width=223) @@ -2645,7 +2643,7 @@ Stage-0 Group By Operator [GBY_14] (rows=1 width=24) Output:["_col0","_col1","_col2"],aggregations:["count()","count(_col0)"],keys:_col1 Select Operator [SEL_12] (rows=1 width=114) - Output:["_col1","_col0"] + Output:["_col0","_col1"] Filter Operator [FIL_45] (rows=1 width=114) predicate:((_col1 = _col1) and ((_col2 - _col1) > 600.0)) Group By Operator [GBY_10] (rows=5 width=114) @@ -5041,7 +5039,7 @@ Stage-4 SHUFFLE [RS_14] PartitionCols:_col2 Select Operator [SEL_13] (rows=26 width=491) - Output:["_col1","_col2","_col5","sum_window_0"] + Output:["sum_window_0","_col1","_col2","_col5"] PTF Operator [PTF_12] (rows=26 width=491) Function definitions:[{},{"name:":"windowingtablefunction","order by:":"_col5 ASC NULLS FIRST","partition by:":"_col2"}] Select Operator [SEL_11] (rows=26 width=491)