Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 0DC9E200CC0 for ; Sat, 24 Jun 2017 11:49:27 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 0C871160BF9; Sat, 24 Jun 2017 09:49:27 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id A98B6160BFC for ; Sat, 24 Jun 2017 11:49:24 +0200 (CEST) Received: (qmail 36192 invoked by uid 500); 24 Jun 2017 09:49:22 -0000 Mailing-List: contact commits-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hive-dev@hive.apache.org Delivered-To: mailing list commits@hive.apache.org Received: (qmail 35376 invoked by uid 99); 24 Jun 2017 09:49:22 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 24 Jun 2017 09:49:22 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 71C9FDFE22; Sat, 24 Jun 2017 09:49:19 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: jcamacho@apache.org To: commits@hive.apache.org Date: Sat, 24 Jun 2017 09:49:31 -0000 Message-Id: <7836d8a0550c482aa334af5e98ef8877@git.apache.org> In-Reply-To: <525572d167554056b534e69546c01fe8@git.apache.org> References: <525572d167554056b534e69546c01fe8@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [13/17] hive git commit: HIVE-16934: Transform COUNT(x) into COUNT() when x is not nullable (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan) archived-at: Sat, 24 Jun 2017 09:49:27 -0000 http://git-wip-us.apache.org/repos/asf/hive/blob/4ad4ceb6/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out index 8bac1f5..3de40ce 100644 --- a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out +++ b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out @@ -3274,11 +3274,10 @@ STAGE PLANS: Tez #### A masked pattern was here #### Edges: - Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 6 (SIMPLE_EDGE) Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE) Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE), Union 6 (CONTAINS) - Reducer 7 <- Union 6 (SIMPLE_EDGE) - Reducer 9 <- Map 8 (CUSTOM_SIMPLE_EDGE), Union 6 (CONTAINS) + Reducer 8 <- Map 7 (CUSTOM_SIMPLE_EDGE), Union 6 (CONTAINS) #### A masked pattern was here #### Vertices: Map 1 @@ -3318,7 +3317,7 @@ STAGE PLANS: value expressions: _col0 (type: string) Execution mode: llap LLAP IO: no inputs - Map 8 + Map 7 Map Operator Tree: TableScan alias: srcpart @@ -3343,7 +3342,7 @@ STAGE PLANS: Reduce Operator Tree: Merge Join Operator condition map: - Inner Join 0 to 1 + Left Semi Join 0 to 1 keys: 0 _col0 (type: string) 1 _col0 (type: string) @@ -3393,35 +3392,22 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Reducer 7 - Execution mode: llap - Reduce Operator Tree: - Group By Operator - keys: KEY._col0 (type: string) - mode: mergepartial - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Select Operator - expressions: _col0 (type: string) - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Dynamic Partitioning Event Operator - Target column: ds (string) - Target Input: srcpart - Partition key expr: ds + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Target Vertex: Map 1 - Reducer 9 + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Dynamic Partitioning Event Operator + Target column: ds (string) + Target Input: srcpart + Partition key expr: ds + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Target Vertex: Map 1 + Reducer 8 Execution mode: llap Reduce Operator Tree: Group By Operator @@ -3442,6 +3428,21 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Dynamic Partitioning Event Operator + Target column: ds (string) + Target Input: srcpart + Partition key expr: ds + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Target Vertex: Map 1 Union 6 Vertex: Union 6 @@ -3481,11 +3482,10 @@ STAGE PLANS: Tez #### A masked pattern was here #### Edges: - Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 6 (SIMPLE_EDGE) Reducer 3 <- Reducer 2 (SIMPLE_EDGE) Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE), Union 6 (CONTAINS) - Reducer 7 <- Union 6 (SIMPLE_EDGE) - Reducer 9 <- Map 8 (CUSTOM_SIMPLE_EDGE), Union 6 (CONTAINS) + Reducer 8 <- Map 7 (CUSTOM_SIMPLE_EDGE), Union 6 (CONTAINS) #### A masked pattern was here #### Vertices: Map 1 @@ -3525,7 +3525,7 @@ STAGE PLANS: value expressions: _col0 (type: string) Execution mode: llap LLAP IO: no inputs - Map 8 + Map 7 Map Operator Tree: TableScan alias: srcpart @@ -3550,7 +3550,7 @@ STAGE PLANS: Reduce Operator Tree: Merge Join Operator condition map: - Inner Join 0 to 1 + Left Semi Join 0 to 1 keys: 0 _col0 (type: string) 1 _col0 (type: string) @@ -3602,35 +3602,22 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Reducer 7 - Execution mode: llap - Reduce Operator Tree: - Group By Operator - keys: KEY._col0 (type: string) - mode: mergepartial - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Select Operator - expressions: _col0 (type: string) - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Dynamic Partitioning Event Operator - Target column: ds (string) - Target Input: srcpart - Partition key expr: ds + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Target Vertex: Map 1 - Reducer 9 + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Dynamic Partitioning Event Operator + Target column: ds (string) + Target Input: srcpart + Partition key expr: ds + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Target Vertex: Map 1 + Reducer 8 Execution mode: llap Reduce Operator Tree: Group By Operator @@ -3651,6 +3638,21 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Dynamic Partitioning Event Operator + Target column: ds (string) + Target Input: srcpart + Partition key expr: ds + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Target Vertex: Map 1 Union 6 Vertex: Union 6 @@ -5654,10 +5656,10 @@ STAGE PLANS: Tez #### A masked pattern was here #### Edges: - Reducer 3 <- Map 2 (CUSTOM_SIMPLE_EDGE), Union 4 (CONTAINS) - Reducer 5 <- Map 1 (BROADCAST_EDGE), Union 4 (SIMPLE_EDGE) - Reducer 6 <- Reducer 5 (SIMPLE_EDGE) - Reducer 8 <- Map 7 (CUSTOM_SIMPLE_EDGE), Union 4 (CONTAINS) + Map 1 <- Union 5 (BROADCAST_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE) + Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE), Union 5 (CONTAINS) + Reducer 7 <- Map 6 (CUSTOM_SIMPLE_EDGE), Union 5 (CONTAINS) #### A masked pattern was here #### Vertices: Map 1 @@ -5670,14 +5672,29 @@ STAGE PLANS: expressions: ds (type: string) outputColumnNames: _col0 Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE + Map Join Operator + condition map: + Left Semi Join 0 to 1 + keys: + 0 _col0 (type: string) + 1 _col0 (type: string) + outputColumnNames: _col0 + input vertices: + 1 Union 5 + Statistics: Num rows: 1000 Data size: 184000 Basic stats: COMPLETE Column stats: COMPLETE + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE Execution mode: llap LLAP IO: no inputs - Map 2 + Map 3 Map Operator Tree: TableScan alias: srcpart @@ -5697,7 +5714,7 @@ STAGE PLANS: value expressions: _col0 (type: string) Execution mode: llap LLAP IO: no inputs - Map 7 + Map 6 Map Operator Tree: TableScan alias: srcpart @@ -5717,45 +5734,32 @@ STAGE PLANS: value expressions: _col0 (type: string) Execution mode: llap LLAP IO: no inputs - Reducer 3 + Reducer 2 Execution mode: llap Reduce Operator Tree: Group By Operator - aggregations: max(VALUE._col0) + keys: KEY._col0 (type: string) mode: mergepartial outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Filter Operator - predicate: _col0 is not null (type: boolean) + File Output Operator + compressed: false Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Reducer 5 + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe + Reducer 4 Execution mode: llap Reduce Operator Tree: Group By Operator - keys: KEY._col0 (type: string) + aggregations: max(VALUE._col0) mode: mergepartial outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Map Join Operator - condition map: - Inner Join 0 to 1 - keys: - 0 _col0 (type: string) - 1 _col0 (type: string) - outputColumnNames: _col0 - input vertices: - 0 Map 1 - Statistics: Num rows: 1000 Data size: 184000 Basic stats: COMPLETE Column stats: COMPLETE + Filter Operator + predicate: _col0 is not null (type: boolean) + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator keys: _col0 (type: string) mode: hash @@ -5766,22 +5770,22 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Reducer 6 - Execution mode: llap - Reduce Operator Tree: - Group By Operator - keys: KEY._col0 (type: string) - mode: mergepartial - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - File Output Operator - compressed: false - Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - table: - input format: org.apache.hadoop.mapred.SequenceFileInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - Reducer 8 + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Dynamic Partitioning Event Operator + Target column: ds (string) + Target Input: srcpart + Partition key expr: ds + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Target Vertex: Map 1 + Reducer 7 Execution mode: llap Reduce Operator Tree: Group By Operator @@ -5802,8 +5806,23 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE - Union 4 - Vertex: Union 4 + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Dynamic Partitioning Event Operator + Target column: ds (string) + Target Input: srcpart + Partition key expr: ds + Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE + Target Vertex: Map 1 + Union 5 + Vertex: Union 5 Stage: Stage-0 Fetch Operator http://git-wip-us.apache.org/repos/asf/hive/blob/4ad4ceb6/ql/src/test/results/clientpositive/llap/except_distinct.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/except_distinct.q.out b/ql/src/test/results/clientpositive/llap/except_distinct.q.out index 4dd596e..7d2222b 100644 --- a/ql/src/test/results/clientpositive/llap/except_distinct.q.out +++ b/ql/src/test/results/clientpositive/llap/except_distinct.q.out @@ -218,11 +218,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(2) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -241,11 +241,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -384,11 +384,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(2) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -398,24 +398,12 @@ STAGE PLANS: Map-reduce partition columns: _col0 (type: string), _col1 (type: string) Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE value expressions: _col2 (type: bigint) - Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 - Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: _col0 (type: string), _col1 (type: string) sort order: ++ Map-reduce partition columns: _col0 (type: string), _col1 (type: string) Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE value expressions: _col2 (type: bigint) - Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 - Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: _col0 (type: string), _col1 (type: string) sort order: ++ @@ -431,11 +419,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -551,7 +539,7 @@ STAGE PLANS: outputColumnNames: _col0, _col1 Statistics: Num rows: 41 Data size: 7954 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(2) + aggregations: count() keys: _col0 (type: string), _col1 (type: string) mode: complete outputColumnNames: _col0, _col1, _col2 @@ -593,7 +581,7 @@ STAGE PLANS: outputColumnNames: _col0, _col1 Statistics: Num rows: 24 Data size: 4656 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(2) + aggregations: count() keys: _col0 (type: string), _col1 (type: string) mode: complete outputColumnNames: _col0, _col1, _col2 @@ -760,7 +748,7 @@ STAGE PLANS: outputColumnNames: _col0 Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: count(2) + aggregations: count() keys: _col0 (type: int) mode: complete outputColumnNames: _col0, _col1 @@ -817,7 +805,7 @@ STAGE PLANS: outputColumnNames: _col0 Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: count(1) + aggregations: count() keys: _col0 (type: int) mode: complete outputColumnNames: _col0, _col1 http://git-wip-us.apache.org/repos/asf/hive/blob/4ad4ceb6/ql/src/test/results/clientpositive/llap/explainuser_1.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out index 64a4821..e78ceb3 100644 --- a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out +++ b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out @@ -164,13 +164,13 @@ Stage-0 limit:-1 Stage-1 Reducer 2 llap - File Output Operator [FS_8] - Group By Operator [GBY_6] (rows=1 width=8) + File Output Operator [FS_7] + Group By Operator [GBY_5] (rows=1 width=8) Output:["_col0"],aggregations:["count(VALUE._col0)"] <-Map 1 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_5] - Group By Operator [GBY_4] (rows=1 width=8) - Output:["_col0"],aggregations:["count(1)"] + PARTITION_ONLY_SHUFFLE [RS_4] + Group By Operator [GBY_3] (rows=1 width=8) + Output:["_col0"],aggregations:["count()"] Select Operator [SEL_2] (rows=500 width=102) TableScan [TS_0] (rows=500 width=102) default@src_orc_merge_test_part,src_orc_merge_test_part,Tbl:COMPLETE,Col:COMPLETE @@ -862,7 +862,7 @@ Stage-0 Reduce Output Operator [RS_25] PartitionCols:_col0 Group By Operator [GBY_24] (rows=1 width=95) - Output:["_col0","_col1"],aggregations:["count(1)"],keys:_col0 + Output:["_col0","_col1"],aggregations:["count()"],keys:_col0 Select Operator [SEL_5] (rows=1 width=87) Output:["_col0"] Group By Operator [GBY_4] (rows=1 width=8) @@ -879,7 +879,7 @@ Stage-0 Reduce Output Operator [RS_25] PartitionCols:_col0 Group By Operator [GBY_24] (rows=1 width=95) - Output:["_col0","_col1"],aggregations:["count(1)"],keys:_col0 + Output:["_col0","_col1"],aggregations:["count()"],keys:_col0 Select Operator [SEL_12] (rows=1 width=87) Output:["_col0"] Group By Operator [GBY_11] (rows=1 width=8) @@ -896,7 +896,7 @@ Stage-0 Reduce Output Operator [RS_25] PartitionCols:_col0 Group By Operator [GBY_24] (rows=1 width=95) - Output:["_col0","_col1"],aggregations:["count(1)"],keys:_col0 + Output:["_col0","_col1"],aggregations:["count()"],keys:_col0 Select Operator [SEL_20] (rows=1 width=87) Output:["_col0"] Group By Operator [GBY_19] (rows=1 width=8) @@ -2061,76 +2061,72 @@ POSTHOOK: type: QUERY Plan optimized by CBO. Vertex dependency in root stage -Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE) +Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE) Reducer 3 <- Reducer 2 (SIMPLE_EDGE) -Reducer 4 <- Reducer 3 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE) +Reducer 4 <- Reducer 3 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE) Reducer 6 <- Map 5 (SIMPLE_EDGE) -Reducer 7 <- Map 5 (SIMPLE_EDGE) Stage-0 Fetch Operator limit:-1 Stage-1 Reducer 4 llap - File Output Operator [FS_33] - Merge Join Operator [MERGEJOIN_46] (rows=34 width=186) - Conds:RS_29._col2=RS_30._col0(Left Semi),Output:["_col0","_col1","_col2"] + File Output Operator [FS_31] + Merge Join Operator [MERGEJOIN_44] (rows=34 width=186) + Conds:RS_27._col2=RS_28._col0(Left Semi),Output:["_col0","_col1","_col2"] <-Reducer 3 [SIMPLE_EDGE] llap - SHUFFLE [RS_29] + SHUFFLE [RS_27] PartitionCols:_col2 - Filter Operator [FIL_39] (rows=83 width=186) + Filter Operator [FIL_37] (rows=83 width=186) predicate:_col2 is not null - Group By Operator [GBY_16] (rows=83 width=186) + Group By Operator [GBY_14] (rows=83 width=186) Output:["_col0","_col1","_col2"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1 <-Reducer 2 [SIMPLE_EDGE] llap - SHUFFLE [RS_15] + SHUFFLE [RS_13] PartitionCols:_col0, _col1 - Group By Operator [GBY_14] (rows=83 width=186) + Group By Operator [GBY_12] (rows=83 width=186) Output:["_col0","_col1","_col2"],aggregations:["count()"],keys:_col0, _col1 - Merge Join Operator [MERGEJOIN_45] (rows=166 width=178) - Conds:RS_10._col0=RS_11._col0(Inner),Output:["_col0","_col1"] + Merge Join Operator [MERGEJOIN_43] (rows=166 width=178) + Conds:RS_8._col0=RS_9._col0(Left Semi),Output:["_col0","_col1"] + <-Map 5 [SIMPLE_EDGE] llap + SHUFFLE [RS_9] + PartitionCols:_col0 + Group By Operator [GBY_7] (rows=69 width=87) + Output:["_col0"],keys:_col0 + Select Operator [SEL_5] (rows=166 width=87) + Output:["_col0"] + Filter Operator [FIL_39] (rows=166 width=87) + predicate:(key > '8') + TableScan [TS_3] (rows=500 width=87) + default@src_cbo,src_cbo,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] <-Map 1 [SIMPLE_EDGE] llap - SHUFFLE [RS_10] + SHUFFLE [RS_8] PartitionCols:_col0 Select Operator [SEL_2] (rows=166 width=178) Output:["_col0","_col1"] - Filter Operator [FIL_40] (rows=166 width=178) + Filter Operator [FIL_38] (rows=166 width=178) predicate:(key > '8') TableScan [TS_0] (rows=500 width=178) default@src_cbo,b,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"] - <-Reducer 6 [SIMPLE_EDGE] llap - SHUFFLE [RS_11] - PartitionCols:_col0 - Group By Operator [GBY_8] (rows=69 width=87) - Output:["_col0"],keys:KEY._col0 - <-Map 5 [SIMPLE_EDGE] llap - SHUFFLE [RS_7] - PartitionCols:_col0 - Group By Operator [GBY_6] (rows=69 width=87) - Output:["_col0"],keys:key - Filter Operator [FIL_41] (rows=166 width=87) - predicate:(key > '8') - TableScan [TS_3] (rows=500 width=87) - default@src_cbo,src_cbo,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] - <-Reducer 7 [SIMPLE_EDGE] llap - SHUFFLE [RS_30] + <-Reducer 6 [SIMPLE_EDGE] llap + SHUFFLE [RS_28] PartitionCols:_col0 - Group By Operator [GBY_28] (rows=34 width=8) + Group By Operator [GBY_26] (rows=34 width=8) Output:["_col0"],keys:_col0 - Select Operator [SEL_26] (rows=69 width=8) + Select Operator [SEL_24] (rows=69 width=8) Output:["_col0"] - Filter Operator [FIL_42] (rows=69 width=8) + Filter Operator [FIL_40] (rows=69 width=8) predicate:_col1 is not null - Select Operator [SEL_44] (rows=69 width=8) + Select Operator [SEL_42] (rows=69 width=8) Output:["_col1"] - Group By Operator [GBY_24] (rows=69 width=95) + Group By Operator [GBY_22] (rows=69 width=95) Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0 <-Map 5 [SIMPLE_EDGE] llap - SHUFFLE [RS_23] + SHUFFLE [RS_21] PartitionCols:_col0 - Group By Operator [GBY_22] (rows=69 width=95) + Group By Operator [GBY_20] (rows=69 width=95) Output:["_col0","_col1"],aggregations:["count()"],keys:key - Filter Operator [FIL_43] (rows=166 width=87) + Filter Operator [FIL_41] (rows=166 width=87) predicate:(key > '9') Please refer to the previous TableScan [TS_3] @@ -2674,30 +2670,30 @@ Stage-0 limit:-1 Stage-1 Reducer 4 llap - File Output Operator [FS_20] - Group By Operator [GBY_18] (rows=1 width=16) + File Output Operator [FS_19] + Group By Operator [GBY_17] (rows=1 width=16) Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"] <-Reducer 3 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_17] - Group By Operator [GBY_16] (rows=1 width=16) + PARTITION_ONLY_SHUFFLE [RS_16] + Group By Operator [GBY_15] (rows=1 width=16) Output:["_col0","_col1"],aggregations:["sum(_col0)","sum(_col1)"] - Select Operator [SEL_14] (rows=14 width=94) + Select Operator [SEL_13] (rows=14 width=94) Output:["_col0","_col1"] - Group By Operator [GBY_13] (rows=14 width=94) + Group By Operator [GBY_12] (rows=14 width=94) Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0 <-Reducer 2 [SIMPLE_EDGE] llap - SHUFFLE [RS_12] + SHUFFLE [RS_11] PartitionCols:_col0 - Group By Operator [GBY_11] (rows=14 width=94) - Output:["_col0","_col1"],aggregations:["count(1)"],keys:_col0 - Merge Join Operator [MERGEJOIN_25] (rows=60 width=86) + Group By Operator [GBY_10] (rows=14 width=94) + Output:["_col0","_col1"],aggregations:["count()"],keys:_col0 + Merge Join Operator [MERGEJOIN_24] (rows=60 width=86) Conds:RS_6._col0=RS_7._col0(Inner),Output:["_col0"] <-Map 1 [SIMPLE_EDGE] llap SHUFFLE [RS_6] PartitionCols:_col0 Select Operator [SEL_2] (rows=25 width=86) Output:["_col0"] - Filter Operator [FIL_23] (rows=25 width=86) + Filter Operator [FIL_22] (rows=25 width=86) predicate:key is not null TableScan [TS_0] (rows=25 width=86) default@src1,x,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] @@ -2706,7 +2702,7 @@ Stage-0 PartitionCols:_col0 Select Operator [SEL_5] (rows=500 width=87) Output:["_col0"] - Filter Operator [FIL_24] (rows=500 width=87) + Filter Operator [FIL_23] (rows=500 width=87) predicate:key is not null TableScan [TS_3] (rows=500 width=87) default@src,y,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] @@ -2735,30 +2731,30 @@ Stage-0 limit:-1 Stage-1 Reducer 4 llap - File Output Operator [FS_20] - Group By Operator [GBY_18] (rows=1 width=16) + File Output Operator [FS_19] + Group By Operator [GBY_17] (rows=1 width=16) Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"] <-Reducer 3 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_17] - Group By Operator [GBY_16] (rows=1 width=16) + PARTITION_ONLY_SHUFFLE [RS_16] + Group By Operator [GBY_15] (rows=1 width=16) Output:["_col0","_col1"],aggregations:["sum(_col0)","sum(_col1)"] - Select Operator [SEL_14] (rows=14 width=94) + Select Operator [SEL_13] (rows=14 width=94) Output:["_col0","_col1"] - Group By Operator [GBY_13] (rows=14 width=94) + Group By Operator [GBY_12] (rows=14 width=94) Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0 <-Reducer 2 [SIMPLE_EDGE] llap - SHUFFLE [RS_12] + SHUFFLE [RS_11] PartitionCols:_col0 - Group By Operator [GBY_11] (rows=14 width=94) - Output:["_col0","_col1"],aggregations:["count(1)"],keys:_col0 - Merge Join Operator [MERGEJOIN_25] (rows=60 width=86) + Group By Operator [GBY_10] (rows=14 width=94) + Output:["_col0","_col1"],aggregations:["count()"],keys:_col0 + Merge Join Operator [MERGEJOIN_24] (rows=60 width=86) Conds:RS_6._col0=RS_7._col0(Inner),Output:["_col0"] <-Map 1 [SIMPLE_EDGE] llap SHUFFLE [RS_6] PartitionCols:_col0 Select Operator [SEL_2] (rows=25 width=86) Output:["_col0"] - Filter Operator [FIL_23] (rows=25 width=86) + Filter Operator [FIL_22] (rows=25 width=86) predicate:key is not null TableScan [TS_0] (rows=25 width=86) default@src1,x,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] @@ -2767,7 +2763,7 @@ Stage-0 PartitionCols:_col0 Select Operator [SEL_5] (rows=500 width=87) Output:["_col0"] - Filter Operator [FIL_24] (rows=500 width=87) + Filter Operator [FIL_23] (rows=500 width=87) predicate:key is not null TableScan [TS_3] (rows=500 width=87) default@src,y,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] @@ -2796,36 +2792,36 @@ Stage-0 limit:-1 Stage-1 Reducer 4 llap - File Output Operator [FS_20] - Group By Operator [GBY_18] (rows=1 width=16) + File Output Operator [FS_19] + Group By Operator [GBY_17] (rows=1 width=16) Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"] <-Reducer 3 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_17] - Group By Operator [GBY_16] (rows=1 width=16) + PARTITION_ONLY_SHUFFLE [RS_16] + Group By Operator [GBY_15] (rows=1 width=16) Output:["_col0","_col1"],aggregations:["sum(_col0)","sum(_col1)"] - Select Operator [SEL_14] (rows=14 width=94) + Select Operator [SEL_13] (rows=14 width=94) Output:["_col0","_col1"] - Group By Operator [GBY_13] (rows=14 width=94) + Group By Operator [GBY_12] (rows=14 width=94) Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0 <-Map 2 [SIMPLE_EDGE] llap - SHUFFLE [RS_12] + SHUFFLE [RS_11] PartitionCols:_col0 - Group By Operator [GBY_11] (rows=14 width=94) - Output:["_col0","_col1"],aggregations:["count(1)"],keys:_col0 - Map Join Operator [MAPJOIN_25] (rows=60 width=86) + Group By Operator [GBY_10] (rows=14 width=94) + Output:["_col0","_col1"],aggregations:["count()"],keys:_col0 + Map Join Operator [MAPJOIN_24] (rows=60 width=86) Conds:RS_6._col0=SEL_5._col0(Inner),Output:["_col0"] <-Map 1 [BROADCAST_EDGE] llap BROADCAST [RS_6] PartitionCols:_col0 Select Operator [SEL_2] (rows=25 width=86) Output:["_col0"] - Filter Operator [FIL_23] (rows=25 width=86) + Filter Operator [FIL_22] (rows=25 width=86) predicate:key is not null TableScan [TS_0] (rows=25 width=86) default@src1,x,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] <-Select Operator [SEL_5] (rows=500 width=87) Output:["_col0"] - Filter Operator [FIL_24] (rows=500 width=87) + Filter Operator [FIL_23] (rows=500 width=87) predicate:key is not null TableScan [TS_3] (rows=500 width=87) default@src,y,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] @@ -2854,30 +2850,30 @@ Stage-0 limit:-1 Stage-1 Reducer 4 llap - File Output Operator [FS_22] - Group By Operator [GBY_20] (rows=1 width=16) + File Output Operator [FS_21] + Group By Operator [GBY_19] (rows=1 width=16) Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"] <-Reducer 3 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_19] - Group By Operator [GBY_18] (rows=1 width=16) + PARTITION_ONLY_SHUFFLE [RS_18] + Group By Operator [GBY_17] (rows=1 width=16) Output:["_col0","_col1"],aggregations:["sum(_col0)","sum(_col1)"] - Select Operator [SEL_16] (rows=12 width=94) + Select Operator [SEL_15] (rows=12 width=94) Output:["_col0","_col1"] - Group By Operator [GBY_15] (rows=12 width=94) + Group By Operator [GBY_14] (rows=12 width=94) Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0 <-Reducer 2 [SIMPLE_EDGE] llap - SHUFFLE [RS_14] + SHUFFLE [RS_13] PartitionCols:_col0 - Group By Operator [GBY_13] (rows=12 width=94) - Output:["_col0","_col1"],aggregations:["count(1)"],keys:_col0 - Merge Join Operator [MERGEJOIN_27] (rows=25 width=86) + Group By Operator [GBY_12] (rows=12 width=94) + Output:["_col0","_col1"],aggregations:["count()"],keys:_col0 + Merge Join Operator [MERGEJOIN_26] (rows=25 width=86) Conds:RS_8._col0=RS_9._col0(Left Semi),Output:["_col0"] <-Map 1 [SIMPLE_EDGE] llap SHUFFLE [RS_8] PartitionCols:_col0 Select Operator [SEL_2] (rows=25 width=86) Output:["_col0"] - Filter Operator [FIL_25] (rows=25 width=86) + Filter Operator [FIL_24] (rows=25 width=86) predicate:key is not null TableScan [TS_0] (rows=25 width=86) default@src1,x,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] @@ -2888,7 +2884,7 @@ Stage-0 Output:["_col0"],keys:_col0 Select Operator [SEL_5] (rows=500 width=87) Output:["_col0"] - Filter Operator [FIL_26] (rows=500 width=87) + Filter Operator [FIL_25] (rows=500 width=87) predicate:key is not null TableScan [TS_3] (rows=500 width=87) default@src,y,Tbl:COMPLETE,Col:COMPLETE,Output:["key"] @@ -3050,11 +3046,11 @@ Stage-0 limit:-1 Stage-1 Reducer 2 llap - File Output Operator [FS_6] - Group By Operator [GBY_4] (rows=1 width=8) - Output:["_col0"],aggregations:["count(1)"] + File Output Operator [FS_5] + Group By Operator [GBY_3] (rows=1 width=8) + Output:["_col0"],aggregations:["count()"] <-Map 1 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_3] + PARTITION_ONLY_SHUFFLE [RS_2] Select Operator [SEL_1] (rows=5 width=6) TableScan [TS_0] (rows=5 width=6) default@tgt_rc_merge_test,tgt_rc_merge_test,Tbl:COMPLETE,Col:COMPLETE @@ -3122,11 +3118,11 @@ Stage-0 limit:-1 Stage-1 Reducer 2 llap - File Output Operator [FS_6] - Group By Operator [GBY_4] (rows=1 width=8) - Output:["_col0"],aggregations:["count(1)"] + File Output Operator [FS_5] + Group By Operator [GBY_3] (rows=1 width=8) + Output:["_col0"],aggregations:["count()"] <-Map 1 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_3] + PARTITION_ONLY_SHUFFLE [RS_2] Select Operator [SEL_1] (rows=5 width=6) TableScan [TS_0] (rows=5 width=6) default@tgt_rc_merge_test,tgt_rc_merge_test,Tbl:COMPLETE,Col:COMPLETE @@ -5670,30 +5666,30 @@ Stage-0 limit:-1 Stage-1 Reducer 3 llap - File Output Operator [FS_16] - Group By Operator [GBY_14] (rows=1 width=8) + File Output Operator [FS_15] + Group By Operator [GBY_13] (rows=1 width=8) Output:["_col0"],aggregations:["count(VALUE._col0)"] <-Reducer 2 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_13] - Group By Operator [GBY_12] (rows=1 width=8) - Output:["_col0"],aggregations:["count(1)"] + PARTITION_ONLY_SHUFFLE [RS_12] + Group By Operator [GBY_11] (rows=1 width=8) + Output:["_col0"],aggregations:["count()"] <-Map 1 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_11] + PARTITION_ONLY_SHUFFLE [RS_10] PartitionCols:rand() - Map Join Operator [MAPJOIN_21] (rows=1 width=33) + Map Join Operator [MAPJOIN_20] (rows=1 width=33) Conds:SEL_2._col0=RS_7._col0(Inner) <-Map 4 [BROADCAST_EDGE] llap BROADCAST [RS_7] PartitionCols:_col0 Select Operator [SEL_5] (rows=1 width=30) Output:["_col0"] - Filter Operator [FIL_20] (rows=1 width=30) + Filter Operator [FIL_19] (rows=1 width=30) predicate:key is not null TableScan [TS_3] (rows=1 width=30) default@t1,b,Tbl:COMPLETE,Col:NONE,Output:["key"] <-Select Operator [SEL_2] (rows=1 width=30) Output:["_col0"] - Filter Operator [FIL_19] (rows=1 width=30) + Filter Operator [FIL_18] (rows=1 width=30) predicate:key is not null TableScan [TS_0] (rows=1 width=30) default@t1,a,Tbl:COMPLETE,Col:NONE,Output:["key"] http://git-wip-us.apache.org/repos/asf/hive/blob/4ad4ceb6/ql/src/test/results/clientpositive/llap/explainuser_2.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out b/ql/src/test/results/clientpositive/llap/explainuser_2.q.out index 42d5f85..f8a6526 100644 --- a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out +++ b/ql/src/test/results/clientpositive/llap/explainuser_2.q.out @@ -3224,52 +3224,52 @@ Stage-4 Dependency Collection{} Stage-2 Reducer 5 llap - File Output Operator [FS_22] + File Output Operator [FS_21] table:{"name:":"default.dest1"} - Select Operator [SEL_20] (rows=1 width=272) + Select Operator [SEL_19] (rows=1 width=272) Output:["_col0","_col1"] - Group By Operator [GBY_19] (rows=1 width=96) + Group By Operator [GBY_18] (rows=1 width=96) Output:["_col0","_col1"],aggregations:["count(DISTINCT KEY._col1:0._col0)"],keys:KEY._col0 <-Reducer 4 [SIMPLE_EDGE] llap - SHUFFLE [RS_18] + SHUFFLE [RS_17] PartitionCols:_col0 - Group By Operator [GBY_17] (rows=1 width=280) + Group By Operator [GBY_16] (rows=1 width=280) Output:["_col0","_col1","_col2"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, substr(_col1, 5) - Group By Operator [GBY_14] (rows=1 width=272) + Group By Operator [GBY_13] (rows=1 width=272) Output:["_col0","_col1"],keys:KEY._col0, KEY._col1 <-Union 3 [SIMPLE_EDGE] <-Map 6 [CONTAINS] llap - Reduce Output Operator [RS_13] + Reduce Output Operator [RS_12] PartitionCols:_col0, _col1 - Group By Operator [GBY_12] (rows=1 width=272) + Group By Operator [GBY_11] (rows=1 width=272) Output:["_col0","_col1"],keys:_col0, _col1 - Select Operator [SEL_8] (rows=500 width=10) + Select Operator [SEL_7] (rows=500 width=10) Output:["_col0","_col1"] - TableScan [TS_7] (rows=500 width=10) + TableScan [TS_6] (rows=500 width=10) Output:["key","value"] <-Reducer 2 [CONTAINS] llap - Reduce Output Operator [RS_13] + Reduce Output Operator [RS_12] PartitionCols:_col0, _col1 - Group By Operator [GBY_12] (rows=1 width=272) + Group By Operator [GBY_11] (rows=1 width=272) Output:["_col0","_col1"],keys:_col0, _col1 - Select Operator [SEL_6] (rows=1 width=272) + Select Operator [SEL_5] (rows=1 width=272) Output:["_col0","_col1"] - Group By Operator [GBY_5] (rows=1 width=8) + Group By Operator [GBY_4] (rows=1 width=8) Output:["_col0"],aggregations:["count(VALUE._col0)"] <-Map 1 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_4] - Group By Operator [GBY_3] (rows=1 width=8) - Output:["_col0"],aggregations:["count(1)"] + PARTITION_ONLY_SHUFFLE [RS_3] + Group By Operator [GBY_2] (rows=1 width=8) + Output:["_col0"],aggregations:["count()"] Select Operator [SEL_1] (rows=500 width=10) TableScan [TS_0] (rows=500 width=10) default@src,s1,Tbl:COMPLETE,Col:COMPLETE - File Output Operator [FS_29] + File Output Operator [FS_28] table:{"name:":"default.dest2"} - Select Operator [SEL_27] (rows=1 width=456) + Select Operator [SEL_26] (rows=1 width=456) Output:["_col0","_col1","_col2"] - Group By Operator [GBY_26] (rows=1 width=464) + Group By Operator [GBY_25] (rows=1 width=464) Output:["_col0","_col1","_col2"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, _col1 - Please refer to the previous Group By Operator [GBY_14] + Please refer to the previous Group By Operator [GBY_13] Stage-5 Stats-Aggr Operator Stage-1 @@ -3398,68 +3398,68 @@ Stage-4 Dependency Collection{} Stage-2 Reducer 4 llap - File Output Operator [FS_19] + File Output Operator [FS_18] table:{"name:":"default.dest1"} - Select Operator [SEL_17] (rows=1 width=272) + Select Operator [SEL_16] (rows=1 width=272) Output:["_col0","_col1"] - Group By Operator [GBY_16] (rows=1 width=96) + Group By Operator [GBY_15] (rows=1 width=96) Output:["_col0","_col1"],aggregations:["count(DISTINCT KEY._col1:0._col0)"],keys:KEY._col0 <-Union 3 [SIMPLE_EDGE] <-Map 6 [CONTAINS] llap - Reduce Output Operator [RS_15] + Reduce Output Operator [RS_14] PartitionCols:_col0 - Group By Operator [GBY_14] (rows=1 width=280) + Group By Operator [GBY_13] (rows=1 width=280) Output:["_col0","_col1","_col2"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, substr(_col1, 5) - Select Operator [SEL_8] (rows=500 width=10) + Select Operator [SEL_7] (rows=500 width=10) Output:["_col0","_col1"] - TableScan [TS_7] (rows=500 width=10) + TableScan [TS_6] (rows=500 width=10) Output:["key","value"] - Reduce Output Operator [RS_22] + Reduce Output Operator [RS_21] PartitionCols:_col0, _col1 - Group By Operator [GBY_21] (rows=1 width=464) + Group By Operator [GBY_20] (rows=1 width=464) Output:["_col0","_col1","_col2","_col3"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, _col1, substr(_col1, 5) - Please refer to the previous Select Operator [SEL_8] + Please refer to the previous Select Operator [SEL_7] <-Map 7 [CONTAINS] llap - Reduce Output Operator [RS_15] + Reduce Output Operator [RS_14] PartitionCols:_col0 - Group By Operator [GBY_14] (rows=1 width=280) + Group By Operator [GBY_13] (rows=1 width=280) Output:["_col0","_col1","_col2"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, substr(_col1, 5) - Select Operator [SEL_11] (rows=500 width=10) + Select Operator [SEL_10] (rows=500 width=10) Output:["_col0","_col1"] - TableScan [TS_10] (rows=500 width=10) + TableScan [TS_9] (rows=500 width=10) Output:["key","value"] - Reduce Output Operator [RS_22] + Reduce Output Operator [RS_21] PartitionCols:_col0, _col1 - Group By Operator [GBY_21] (rows=1 width=464) + Group By Operator [GBY_20] (rows=1 width=464) Output:["_col0","_col1","_col2","_col3"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, _col1, substr(_col1, 5) - Please refer to the previous Select Operator [SEL_11] + Please refer to the previous Select Operator [SEL_10] <-Reducer 2 [CONTAINS] llap - Reduce Output Operator [RS_15] + Reduce Output Operator [RS_14] PartitionCols:_col0 - Group By Operator [GBY_14] (rows=1 width=280) + Group By Operator [GBY_13] (rows=1 width=280) Output:["_col0","_col1","_col2"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, substr(_col1, 5) - Select Operator [SEL_6] (rows=1 width=272) + Select Operator [SEL_5] (rows=1 width=272) Output:["_col0","_col1"] - Group By Operator [GBY_5] (rows=1 width=8) + Group By Operator [GBY_4] (rows=1 width=8) Output:["_col0"],aggregations:["count(VALUE._col0)"] <-Map 1 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_4] - Group By Operator [GBY_3] (rows=1 width=8) - Output:["_col0"],aggregations:["count(1)"] + PARTITION_ONLY_SHUFFLE [RS_3] + Group By Operator [GBY_2] (rows=1 width=8) + Output:["_col0"],aggregations:["count()"] Select Operator [SEL_1] (rows=500 width=10) TableScan [TS_0] (rows=500 width=10) default@src,s1,Tbl:COMPLETE,Col:COMPLETE - Reduce Output Operator [RS_22] + Reduce Output Operator [RS_21] PartitionCols:_col0, _col1 - Group By Operator [GBY_21] (rows=1 width=464) + Group By Operator [GBY_20] (rows=1 width=464) Output:["_col0","_col1","_col2","_col3"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, _col1, substr(_col1, 5) - Please refer to the previous Select Operator [SEL_6] + Please refer to the previous Select Operator [SEL_5] Reducer 5 llap - File Output Operator [FS_26] + File Output Operator [FS_25] table:{"name:":"default.dest2"} - Select Operator [SEL_24] (rows=1 width=456) + Select Operator [SEL_23] (rows=1 width=456) Output:["_col0","_col1","_col2"] - Group By Operator [GBY_23] (rows=1 width=280) + Group By Operator [GBY_22] (rows=1 width=280) Output:["_col0","_col1","_col2"],aggregations:["count(DISTINCT KEY._col2:0._col0)"],keys:KEY._col0, KEY._col1 <- Please refer to the previous Union 3 [SIMPLE_EDGE] Stage-5 @@ -3506,54 +3506,54 @@ Stage-4 Dependency Collection{} Stage-2 Reducer 4 llap - File Output Operator [FS_17] + File Output Operator [FS_16] table:{"name:":"default.dest1"} - Select Operator [SEL_15] (rows=1 width=272) + Select Operator [SEL_14] (rows=1 width=272) Output:["_col0","_col1"] - Group By Operator [GBY_14] (rows=1 width=96) + Group By Operator [GBY_13] (rows=1 width=96) Output:["_col0","_col1"],aggregations:["count(DISTINCT KEY._col1:0._col0)"],keys:KEY._col0 <-Union 3 [SIMPLE_EDGE] <-Map 6 [CONTAINS] llap - Reduce Output Operator [RS_13] + Reduce Output Operator [RS_12] PartitionCols:_col0 - Group By Operator [GBY_12] (rows=1 width=280) + Group By Operator [GBY_11] (rows=1 width=280) Output:["_col0","_col1","_col2"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, substr(_col1, 5) - Select Operator [SEL_8] (rows=500 width=10) + Select Operator [SEL_7] (rows=500 width=10) Output:["_col0","_col1"] - TableScan [TS_7] (rows=500 width=10) + TableScan [TS_6] (rows=500 width=10) Output:["key","value"] - Reduce Output Operator [RS_20] + Reduce Output Operator [RS_19] PartitionCols:_col0, _col1 - Group By Operator [GBY_19] (rows=1 width=464) + Group By Operator [GBY_18] (rows=1 width=464) Output:["_col0","_col1","_col2","_col3"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, _col1, substr(_col1, 5) - Please refer to the previous Select Operator [SEL_8] + Please refer to the previous Select Operator [SEL_7] <-Reducer 2 [CONTAINS] llap - Reduce Output Operator [RS_13] + Reduce Output Operator [RS_12] PartitionCols:_col0 - Group By Operator [GBY_12] (rows=1 width=280) + Group By Operator [GBY_11] (rows=1 width=280) Output:["_col0","_col1","_col2"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, substr(_col1, 5) - Select Operator [SEL_6] (rows=1 width=272) + Select Operator [SEL_5] (rows=1 width=272) Output:["_col0","_col1"] - Group By Operator [GBY_5] (rows=1 width=8) + Group By Operator [GBY_4] (rows=1 width=8) Output:["_col0"],aggregations:["count(VALUE._col0)"] <-Map 1 [CUSTOM_SIMPLE_EDGE] llap - PARTITION_ONLY_SHUFFLE [RS_4] - Group By Operator [GBY_3] (rows=1 width=8) - Output:["_col0"],aggregations:["count(1)"] + PARTITION_ONLY_SHUFFLE [RS_3] + Group By Operator [GBY_2] (rows=1 width=8) + Output:["_col0"],aggregations:["count()"] Select Operator [SEL_1] (rows=500 width=10) TableScan [TS_0] (rows=500 width=10) default@src,s1,Tbl:COMPLETE,Col:COMPLETE - Reduce Output Operator [RS_20] + Reduce Output Operator [RS_19] PartitionCols:_col0, _col1 - Group By Operator [GBY_19] (rows=1 width=464) + Group By Operator [GBY_18] (rows=1 width=464) Output:["_col0","_col1","_col2","_col3"],aggregations:["count(DISTINCT substr(_col1, 5))"],keys:_col0, _col1, substr(_col1, 5) - Please refer to the previous Select Operator [SEL_6] + Please refer to the previous Select Operator [SEL_5] Reducer 5 llap - File Output Operator [FS_24] + File Output Operator [FS_23] table:{"name:":"default.dest2"} - Select Operator [SEL_22] (rows=1 width=456) + Select Operator [SEL_21] (rows=1 width=456) Output:["_col0","_col1","_col2"] - Group By Operator [GBY_21] (rows=1 width=280) + Group By Operator [GBY_20] (rows=1 width=280) Output:["_col0","_col1","_col2"],aggregations:["count(DISTINCT KEY._col2:0._col0)"],keys:KEY._col0, KEY._col1 <- Please refer to the previous Union 3 [SIMPLE_EDGE] Stage-5 http://git-wip-us.apache.org/repos/asf/hive/blob/4ad4ceb6/ql/src/test/results/clientpositive/llap/intersect_all.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/intersect_all.q.out b/ql/src/test/results/clientpositive/llap/intersect_all.q.out index 2422090..1a3a035 100644 --- a/ql/src/test/results/clientpositive/llap/intersect_all.q.out +++ b/ql/src/test/results/clientpositive/llap/intersect_all.q.out @@ -158,11 +158,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -181,11 +181,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -813,11 +813,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -836,11 +836,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -859,11 +859,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -882,11 +882,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -1598,7 +1598,7 @@ STAGE PLANS: outputColumnNames: _col0 Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: count(1) + aggregations: count() keys: _col0 (type: int) mode: complete outputColumnNames: _col0, _col1 @@ -1654,7 +1654,7 @@ STAGE PLANS: outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: count(1) + aggregations: count() keys: _col0 (type: int) mode: complete outputColumnNames: _col0, _col1 http://git-wip-us.apache.org/repos/asf/hive/blob/4ad4ceb6/ql/src/test/results/clientpositive/llap/intersect_distinct.q.out ---------------------------------------------------------------------- diff --git a/ql/src/test/results/clientpositive/llap/intersect_distinct.q.out b/ql/src/test/results/clientpositive/llap/intersect_distinct.q.out index 6921d53..a871ba3 100644 --- a/ql/src/test/results/clientpositive/llap/intersect_distinct.q.out +++ b/ql/src/test/results/clientpositive/llap/intersect_distinct.q.out @@ -156,11 +156,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -179,11 +179,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -613,11 +613,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -636,11 +636,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -659,11 +659,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -682,11 +682,11 @@ STAGE PLANS: Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: key (type: string), value (type: string) - outputColumnNames: _col0, _col1 + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator - aggregations: count(1) - keys: _col0 (type: string), _col1 (type: string) + aggregations: count() + keys: key (type: string), value (type: string) mode: hash outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE @@ -1200,7 +1200,7 @@ STAGE PLANS: outputColumnNames: _col0 Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: count(1) + aggregations: count() keys: _col0 (type: int) mode: complete outputColumnNames: _col0, _col1 @@ -1249,7 +1249,7 @@ STAGE PLANS: outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: count(1) + aggregations: count() keys: _col0 (type: int) mode: complete outputColumnNames: _col0, _col1