Return-Path: X-Original-To: apmail-hive-commits-archive@www.apache.org Delivered-To: apmail-hive-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AA2FC17AE1 for ; Wed, 11 Feb 2015 17:48:58 +0000 (UTC) Received: (qmail 591 invoked by uid 500); 11 Feb 2015 17:48:49 -0000 Delivered-To: apmail-hive-commits-archive@hive.apache.org Received: (qmail 478 invoked by uid 500); 11 Feb 2015 17:48:49 -0000 Mailing-List: contact commits-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hive-dev@hive.apache.org Delivered-To: mailing list commits@hive.apache.org Received: (qmail 99362 invoked by uid 99); 11 Feb 2015 17:48:49 -0000 Received: from eris.apache.org (HELO hades.apache.org) (140.211.11.105) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Feb 2015 17:48:49 +0000 Received: from hades.apache.org (localhost [127.0.0.1]) by hades.apache.org (ASF Mail Server at hades.apache.org) with ESMTP id EA2CAAC013F for ; Wed, 11 Feb 2015 17:48:48 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1659014 [17/25] - in /hive/branches/parquet: ./ beeline/src/java/org/apache/hive/beeline/ beeline/src/main/resources/ beeline/src/test/org/apache/hive/beeline/ bin/ common/src/java/org/apache/hadoop/hive/conf/ data/conf/ data/conf/spark/ d... Date: Wed, 11 Feb 2015 17:48:41 -0000 To: commits@hive.apache.org From: brock@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20150211174848.EA2CAAC013F@hades.apache.org> Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby11.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby11.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby11.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby11.q.out Wed Feb 11 17:48:36 2015 @@ -43,29 +43,44 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 2) - Reducer 6 <- Map 1 (SORT, 2) - Reducer 3 <- Reducer 5 (GROUP, 2) - Reducer 4 <- Reducer 6 (GROUP, 2) + Reducer 2 <- Map 4 (GROUP PARTITION-LEVEL SORT, 2) + Reducer 3 <- Map 5 (GROUP PARTITION-LEVEL SORT, 2) #### A masked pattern was here #### Vertices: - Map 1 + Map 4 Map Operator Tree: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: key (type: string) - sort order: + - Map-reduce partition columns: key (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: value (type: string), substr(value, 5) (type: string) - Reducer 3 + Reduce Output Operator + key expressions: value (type: string), key (type: string) + sort order: ++ + Map-reduce partition columns: value (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Map 5 + Map Operator Tree: + TableScan + alias: src + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: substr(value, 5) (type: string), key (type: string) + sort order: ++ + Map-reduce partition columns: substr(value, 5) (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0), count(VALUE._col1) + aggregations: count(KEY._col1:0._col0), count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: complete outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -80,12 +95,12 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 - Reducer 4 + Reducer 3 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0), count(VALUE._col1) + aggregations: count(KEY._col1:0._col0), count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: complete outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -100,38 +115,6 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest2 - Reducer 5 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(KEY._col0), count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint), _col2 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(KEY._col0), count(DISTINCT KEY._col0) - keys: VALUE._col1 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint), _col2 (type: bigint) Stage: Stage-0 Move Operator Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8.q.out Wed Feb 11 17:48:36 2015 @@ -39,29 +39,44 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 2) - Reducer 6 <- Map 1 (SORT, 2) - Reducer 3 <- Reducer 5 (GROUP, 2) - Reducer 4 <- Reducer 6 (GROUP, 2) + Reducer 2 <- Map 4 (GROUP PARTITION-LEVEL SORT, 2) + Reducer 3 <- Map 5 (GROUP PARTITION-LEVEL SORT, 2) #### A masked pattern was here #### Vertices: - Map 1 + Map 4 Map Operator Tree: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: substr(value, 5) (type: string) - sort order: + - Map-reduce partition columns: substr(value, 5) (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string) - Reducer 3 + Reduce Output Operator + key expressions: key (type: string), substr(value, 5) (type: string) + sort order: ++ + Map-reduce partition columns: key (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Map 5 + Map Operator Tree: + TableScan + alias: src + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: key (type: string), substr(value, 5) (type: string) + sort order: ++ + Map-reduce partition columns: key (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: complete outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -76,12 +91,12 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 - Reducer 4 + Reducer 3 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: complete outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -96,38 +111,6 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest2 - Reducer 5 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) Stage: Stage-0 Move Operator @@ -828,29 +811,44 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 2) - Reducer 6 <- Map 1 (SORT, 2) - Reducer 3 <- Reducer 5 (GROUP, 2) - Reducer 4 <- Reducer 6 (GROUP, 2) + Reducer 2 <- Map 4 (GROUP PARTITION-LEVEL SORT, 2) + Reducer 3 <- Map 5 (GROUP PARTITION-LEVEL SORT, 2) #### A masked pattern was here #### Vertices: - Map 1 + Map 4 Map Operator Tree: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: substr(value, 5) (type: string) - sort order: + - Map-reduce partition columns: substr(value, 5) (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string) - Reducer 3 + Reduce Output Operator + key expressions: key (type: string), substr(value, 5) (type: string) + sort order: ++ + Map-reduce partition columns: key (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Map 5 + Map Operator Tree: + TableScan + alias: src + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: key (type: string), substr(value, 5) (type: string) + sort order: ++ + Map-reduce partition columns: key (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: complete outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -865,12 +863,12 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 - Reducer 4 + Reducer 3 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: complete outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -885,38 +883,6 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest2 - Reducer 5 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) Stage: Stage-0 Move Operator Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_map.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_map.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_map.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_map.q.out Wed Feb 11 17:48:36 2015 @@ -39,10 +39,7 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 31) - Reducer 6 <- Map 1 (SORT, 31) - Reducer 3 <- Reducer 5 (GROUP, 31) - Reducer 4 <- Reducer 6 (GROUP, 31) + Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 31) #### A masked pattern was here #### Vertices: Map 1 @@ -50,84 +47,55 @@ STAGE PLANS: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: substr(value, 5) (type: string) - sort order: + - Map-reduce partition columns: substr(value, 5) (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string) - Reducer 3 - Reduce Operator Tree: - Group By Operator - aggregations: count(VALUE._col0) - keys: KEY._col0 (type: string) - mode: final - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: UDFToInteger(_col0) (type: int), _col1 (type: bigint) - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - name: default.dest1 - Reducer 4 - Reduce Operator Tree: - Group By Operator - aggregations: count(VALUE._col0) - keys: KEY._col0 (type: string) - mode: final - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: UDFToInteger(_col0) (type: int), _col1 (type: bigint) - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - name: default.dest2 - Reducer 5 + Reduce Output Operator + key expressions: key (type: string), substr(value, 5) (type: string) + sort order: ++ + Map-reduce partition columns: key (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Forward Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash + aggregations: count(DISTINCT KEY._col1:0._col0) + keys: KEY._col0 (type: string) + mode: complete outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: UDFToInteger(_col0) (type: int), _col1 (type: bigint) + outputColumnNames: _col0, _col1 + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + File Output Operator + compressed: false + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + table: + input format: org.apache.hadoop.mapred.TextInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat + serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe + name: default.dest1 Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash + aggregations: count(DISTINCT KEY._col1:0._col0) + keys: KEY._col0 (type: string) + mode: complete outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: UDFToInteger(_col0) (type: int), _col1 (type: bigint) + outputColumnNames: _col0, _col1 + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + File Output Operator + compressed: false + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + table: + input format: org.apache.hadoop.mapred.TextInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat + serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe + name: default.dest2 Stage: Stage-0 Move Operator Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out Wed Feb 11 17:48:36 2015 @@ -39,29 +39,56 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 31) - Reducer 6 <- Map 1 (SORT, 31) - Reducer 3 <- Reducer 5 (GROUP, 31) - Reducer 4 <- Reducer 6 (GROUP, 31) + Reducer 2 <- Map 4 (GROUP PARTITION-LEVEL SORT, 31) + Reducer 3 <- Map 5 (GROUP PARTITION-LEVEL SORT, 31) #### A masked pattern was here #### Vertices: - Map 1 + Map 4 Map Operator Tree: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: substr(value, 5) (type: string) - sort order: + - Map-reduce partition columns: substr(value, 5) (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string) - Reducer 3 + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: key (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string) + sort order: ++ + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Map 5 + Map Operator Tree: + TableScan + alias: src + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: key (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string) + sort order: ++ + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: complete outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -76,12 +103,12 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 - Reducer 4 + Reducer 3 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: complete outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -96,38 +123,6 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest2 - Reducer 5 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) Stage: Stage-0 Move Operator Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out Wed Feb 11 17:48:36 2015 @@ -39,10 +39,7 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 31) - Reducer 6 <- Map 1 (SORT, 31) - Reducer 3 <- Reducer 5 (GROUP, 31) - Reducer 4 <- Reducer 6 (GROUP, 31) + Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 31) #### A masked pattern was here #### Vertices: Map 1 @@ -50,84 +47,55 @@ STAGE PLANS: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: substr(value, 5) (type: string) - sort order: + - Map-reduce partition columns: substr(value, 5) (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string) - Reducer 3 - Reduce Operator Tree: - Group By Operator - aggregations: count(VALUE._col0) - keys: KEY._col0 (type: string) - mode: final - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: UDFToInteger(_col0) (type: int), _col1 (type: bigint) - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - name: default.dest1 - Reducer 4 - Reduce Operator Tree: - Group By Operator - aggregations: count(VALUE._col0) - keys: KEY._col0 (type: string) - mode: final - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: UDFToInteger(_col0) (type: int), _col1 (type: bigint) - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - name: default.dest2 - Reducer 5 + Reduce Output Operator + key expressions: key (type: string), substr(value, 5) (type: string) + sort order: ++ + Map-reduce partition columns: key (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Forward Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash + aggregations: count(DISTINCT KEY._col1:0._col0) + keys: KEY._col0 (type: string) + mode: complete outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: UDFToInteger(_col0) (type: int), _col1 (type: bigint) + outputColumnNames: _col0, _col1 + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + File Output Operator + compressed: false + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + table: + input format: org.apache.hadoop.mapred.TextInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat + serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe + name: default.dest1 Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash + aggregations: count(DISTINCT KEY._col1:0._col0) + keys: KEY._col0 (type: string) + mode: complete outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: UDFToInteger(_col0) (type: int), _col1 (type: bigint) + outputColumnNames: _col0, _col1 + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + File Output Operator + compressed: false + Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE + table: + input format: org.apache.hadoop.mapred.TextInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat + serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe + name: default.dest2 Stage: Stage-0 Move Operator Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby9.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby9.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby9.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby9.q.out Wed Feb 11 17:48:36 2015 @@ -39,29 +39,56 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 2) - Reducer 6 <- Map 1 (SORT, 2) - Reducer 3 <- Reducer 5 (GROUP, 2) - Reducer 4 <- Reducer 6 (GROUP, 2) + Reducer 2 <- Map 4 (GROUP PARTITION-LEVEL SORT, 2) + Reducer 3 <- Map 5 (GROUP PARTITION-LEVEL SORT, 2) #### A masked pattern was here #### Vertices: - Map 1 + Map 4 Map Operator Tree: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: substr(value, 5) (type: string) - sort order: + - Map-reduce partition columns: substr(value, 5) (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string), value (type: string) - Reducer 3 + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: key (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string) + sort order: ++ + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Map 5 + Map Operator Tree: + TableScan + alias: src + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: key (type: string), value (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string) + sort order: +++ + Map-reduce partition columns: _col0 (type: string), _col1 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: mergepartial outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -76,12 +103,12 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 - Reducer 4 + Reducer 3 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col2:0._col0) keys: KEY._col0 (type: string), KEY._col1 (type: string) - mode: final + mode: mergepartial outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -96,38 +123,6 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest2 - Reducer 5 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string), VALUE._col1 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string), _col1 (type: string) - sort order: ++ - Map-reduce partition columns: _col0 (type: string), _col1 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col2 (type: bigint) Stage: Stage-0 Move Operator @@ -829,29 +824,56 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 2) - Reducer 6 <- Map 1 (SORT, 2) - Reducer 3 <- Reducer 5 (GROUP, 2) - Reducer 4 <- Reducer 6 (GROUP, 2) + Reducer 2 <- Map 4 (GROUP PARTITION-LEVEL SORT, 2) + Reducer 3 <- Map 5 (GROUP PARTITION-LEVEL SORT, 2) #### A masked pattern was here #### Vertices: - Map 1 + Map 4 Map Operator Tree: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: substr(value, 5) (type: string) - sort order: + - Map-reduce partition columns: substr(value, 5) (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string), value (type: string) - Reducer 3 + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: key (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string) + sort order: ++ + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Map 5 + Map Operator Tree: + TableScan + alias: src + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: value (type: string), key (type: string) + outputColumnNames: value, key + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: value (type: string), key (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string) + sort order: +++ + Map-reduce partition columns: _col0 (type: string), _col1 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: mergepartial outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -866,12 +888,12 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 - Reducer 4 + Reducer 3 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col2:0._col0) keys: KEY._col0 (type: string), KEY._col1 (type: string) - mode: final + mode: mergepartial outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -886,38 +908,6 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest2 - Reducer 5 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col1 (type: string), VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string), _col1 (type: string) - sort order: ++ - Map-reduce partition columns: _col0 (type: string), _col1 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col2 (type: bigint) Stage: Stage-0 Move Operator @@ -1619,29 +1609,56 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 2) - Reducer 6 <- Map 1 (SORT, 2) - Reducer 3 <- Reducer 5 (GROUP, 2) - Reducer 4 <- Reducer 6 (GROUP, 2) + Reducer 2 <- Map 4 (GROUP PARTITION-LEVEL SORT, 2) + Reducer 3 <- Map 5 (GROUP PARTITION-LEVEL SORT, 2) #### A masked pattern was here #### Vertices: - Map 1 + Map 4 Map Operator Tree: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: substr(value, 5) (type: string) - sort order: + - Map-reduce partition columns: substr(value, 5) (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string), value (type: string) - Reducer 3 + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: key (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string) + sort order: ++ + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Map 5 + Map Operator Tree: + TableScan + alias: src + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: key (type: string), value (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string) + sort order: +++ + Map-reduce partition columns: _col0 (type: string), _col1 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: mergepartial outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -1656,12 +1673,12 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 - Reducer 4 + Reducer 3 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col2:0._col0) keys: KEY._col0 (type: string), KEY._col1 (type: string) - mode: final + mode: mergepartial outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -1676,38 +1693,6 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest2 - Reducer 5 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string), VALUE._col1 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string), _col1 (type: string) - sort order: ++ - Map-reduce partition columns: _col0 (type: string), _col1 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col2 (type: bigint) Stage: Stage-0 Move Operator @@ -3196,29 +3181,56 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 2) - Reducer 6 <- Map 1 (SORT, 2) - Reducer 3 <- Reducer 5 (GROUP, 2) - Reducer 4 <- Reducer 6 (GROUP, 2) + Reducer 2 <- Map 4 (GROUP PARTITION-LEVEL SORT, 2) + Reducer 3 <- Map 5 (GROUP PARTITION-LEVEL SORT, 2) #### A masked pattern was here #### Vertices: - Map 1 + Map 4 Map Operator Tree: TableScan alias: src Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: substr(value, 5) (type: string) - sort order: + - Map-reduce partition columns: substr(value, 5) (type: string) + Select Operator + expressions: key (type: string), value (type: string) + outputColumnNames: key, value Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string), value (type: string) - Reducer 3 + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: key (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string) + sort order: ++ + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Map 5 + Map Operator Tree: + TableScan + alias: src + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: value (type: string), key (type: string) + outputColumnNames: value, key + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Group By Operator + aggregations: count(DISTINCT substr(value, 5)) + keys: value (type: string), key (type: string), substr(value, 5) (type: string) + mode: hash + outputColumnNames: _col0, _col1, _col2, _col3 + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string) + sort order: +++ + Map-reduce partition columns: _col0 (type: string), _col1 (type: string) + Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col1:0._col0) keys: KEY._col0 (type: string) - mode: final + mode: mergepartial outputColumnNames: _col0, _col1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -3233,12 +3245,12 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 - Reducer 4 + Reducer 3 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) + aggregations: count(DISTINCT KEY._col2:0._col0) keys: KEY._col0 (type: string), KEY._col1 (type: string) - mode: final + mode: mergepartial outputColumnNames: _col0, _col1, _col2 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator @@ -3253,38 +3265,6 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest2 - Reducer 5 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col1 (type: string), VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1, _col2 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string), _col1 (type: string) - sort order: ++ - Map-reduce partition columns: _col0 (type: string), _col1 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col2 (type: bigint) Stage: Stage-0 Move Operator Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out Wed Feb 11 17:48:36 2015 @@ -39,191 +39,6 @@ STAGE PLANS: Stage: Stage-2 Spark Edges: - Reducer 5 <- Map 1 (SORT, 2) - Reducer 6 <- Map 1 (SORT, 2) - Reducer 3 <- Reducer 5 (GROUP, 2) - Reducer 4 <- Reducer 6 (GROUP, 2) -#### A masked pattern was here #### - Vertices: - Map 1 - Map Operator Tree: - TableScan - alias: src - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: value (type: string) - sort order: + - Map-reduce partition columns: value (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: key (type: string), (key + key) (type: double) - Reducer 3 - Reduce Operator Tree: - Group By Operator - aggregations: count(VALUE._col0) - keys: KEY._col0 (type: string) - mode: final - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: UDFToInteger(_col0) (type: int), UDFToInteger(_col1) (type: int) - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - name: default.dest1 - Reducer 4 - Reduce Operator Tree: - Group By Operator - aggregations: count(VALUE._col0) - keys: KEY._col0 (type: double) - mode: final - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: UDFToInteger(_col0) (type: int), UDFToInteger(_col1) (type: int) - outputColumnNames: _col0, _col1 - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - name: default.dest2 - Reducer 5 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col0 (type: string) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - Reducer 6 - Reduce Operator Tree: - Forward - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: count(DISTINCT KEY._col0) - keys: VALUE._col1 (type: double) - mode: hash - outputColumnNames: _col0, _col1 - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: double) - sort order: + - Map-reduce partition columns: _col0 (type: double) - Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE - value expressions: _col1 (type: bigint) - - Stage: Stage-0 - Move Operator - tables: - replace: true - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - name: default.dest1 - - Stage: Stage-3 - Stats-Aggr Operator - - Stage: Stage-1 - Move Operator - tables: - replace: true - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - name: default.dest2 - - Stage: Stage-4 - Stats-Aggr Operator - -PREHOOK: query: from src -insert overwrite table dest1 select key, count(distinct value) group by key -insert overwrite table dest2 select key+key, count(distinct value) group by key+key -PREHOOK: type: QUERY -PREHOOK: Input: default@src -PREHOOK: Output: default@dest1 -PREHOOK: Output: default@dest2 -POSTHOOK: query: from src -insert overwrite table dest1 select key, count(distinct value) group by key -insert overwrite table dest2 select key+key, count(distinct value) group by key+key -POSTHOOK: type: QUERY -POSTHOOK: Input: default@src -POSTHOOK: Output: default@dest1 -POSTHOOK: Output: default@dest2 -POSTHOOK: Lineage: dest1.cnt EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ] -POSTHOOK: Lineage: dest1.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] -POSTHOOK: Lineage: dest2.cnt EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ] -POSTHOOK: Lineage: dest2.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] -PREHOOK: query: select * from dest1 where key < 10 -PREHOOK: type: QUERY -PREHOOK: Input: default@dest1 -#### A masked pattern was here #### -POSTHOOK: query: select * from dest1 where key < 10 -POSTHOOK: type: QUERY -POSTHOOK: Input: default@dest1 -#### A masked pattern was here #### -0 1 -2 1 -4 1 -5 1 -8 1 -9 1 -PREHOOK: query: select * from dest2 where key < 20 order by key limit 10 -PREHOOK: type: QUERY -PREHOOK: Input: default@dest2 -#### A masked pattern was here #### -POSTHOOK: query: select * from dest2 where key < 20 order by key limit 10 -POSTHOOK: type: QUERY -POSTHOOK: Input: default@dest2 -#### A masked pattern was here #### -0 1 -10 1 -16 1 -18 1 -4 1 -8 1 -PREHOOK: query: -- no need to spray by distinct key first -explain -from src -insert overwrite table dest1 select key, count(distinct value) group by key -insert overwrite table dest2 select key+key, count(distinct value) group by key+key -PREHOOK: type: QUERY -POSTHOOK: query: -- no need to spray by distinct key first -explain -from src -insert overwrite table dest1 select key, count(distinct value) group by key -insert overwrite table dest2 select key+key, count(distinct value) group by key+key -POSTHOOK: type: QUERY -STAGE DEPENDENCIES: - Stage-2 is a root stage - Stage-0 depends on stages: Stage-2 - Stage-3 depends on stages: Stage-0 - Stage-1 depends on stages: Stage-2 - Stage-4 depends on stages: Stage-1 - -STAGE PLANS: - Stage: Stage-2 - Spark - Edges: Reducer 2 <- Map 4 (GROUP PARTITION-LEVEL SORT, 2) Reducer 3 <- Map 5 (GROUP PARTITION-LEVEL SORT, 2) #### A masked pattern was here #### Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out Wed Feb 11 17:48:36 2015 @@ -1888,7 +1888,7 @@ ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_SUBQUERY - TOK_UNION + TOK_UNIONALL TOK_QUERY TOK_FROM TOK_TABREF @@ -2241,7 +2241,7 @@ ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_SUBQUERY - TOK_UNION + TOK_UNIONALL TOK_QUERY TOK_FROM TOK_TABREF Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out Wed Feb 11 17:48:36 2015 @@ -1960,7 +1960,7 @@ ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_SUBQUERY - TOK_UNION + TOK_UNIONALL TOK_QUERY TOK_FROM TOK_TABREF @@ -2313,7 +2313,7 @@ ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_SUBQUERY - TOK_UNION + TOK_UNIONALL TOK_QUERY TOK_FROM TOK_TABREF Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/join34.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/join34.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/join34.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/join34.q.out Wed Feb 11 17:48:36 2015 @@ -40,7 +40,7 @@ TOK_QUERY TOK_FROM TOK_JOIN TOK_SUBQUERY - TOK_UNION + TOK_UNIONALL TOK_QUERY TOK_FROM TOK_TABREF Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/join35.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/join35.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/join35.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/join35.q.out Wed Feb 11 17:48:36 2015 @@ -40,7 +40,7 @@ TOK_QUERY TOK_FROM TOK_JOIN TOK_SUBQUERY - TOK_UNION + TOK_UNIONALL TOK_QUERY TOK_FROM TOK_TABREF Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/load_dyn_part14.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/load_dyn_part14.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/load_dyn_part14.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/load_dyn_part14.q.out Wed Feb 11 17:48:36 2015 @@ -36,21 +36,21 @@ value string PREHOOK: query: explain insert overwrite table nzhang_part14 partition(value) select key, value from ( - select 'k1' as key, cast(null as string) as value from src limit 2 + select * from (select 'k1' as key, cast(null as string) as value from src limit 2)a union all - select 'k2' as key, '' as value from src limit 2 + select * from (select 'k2' as key, '' as value from src limit 2)b union all - select 'k3' as key, ' ' as value from src limit 2 + select * from (select 'k3' as key, ' ' as value from src limit 2)c ) T PREHOOK: type: QUERY POSTHOOK: query: explain insert overwrite table nzhang_part14 partition(value) select key, value from ( - select 'k1' as key, cast(null as string) as value from src limit 2 + select * from (select 'k1' as key, cast(null as string) as value from src limit 2)a union all - select 'k2' as key, '' as value from src limit 2 + select * from (select 'k2' as key, '' as value from src limit 2)b union all - select 'k3' as key, ' ' as value from src limit 2 + select * from (select 'k3' as key, ' ' as value from src limit 2)c ) T POSTHOOK: type: QUERY STAGE DEPENDENCIES: @@ -175,22 +175,22 @@ STAGE PLANS: PREHOOK: query: insert overwrite table nzhang_part14 partition(value) select key, value from ( - select 'k1' as key, cast(null as string) as value from src limit 2 + select * from (select 'k1' as key, cast(null as string) as value from src limit 2)a union all - select 'k2' as key, '' as value from src limit 2 + select * from (select 'k2' as key, '' as value from src limit 2)b union all - select 'k3' as key, ' ' as value from src limit 2 + select * from (select 'k3' as key, ' ' as value from src limit 2)c ) T PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: default@nzhang_part14 POSTHOOK: query: insert overwrite table nzhang_part14 partition(value) select key, value from ( - select 'k1' as key, cast(null as string) as value from src limit 2 + select * from (select 'k1' as key, cast(null as string) as value from src limit 2)a union all - select 'k2' as key, '' as value from src limit 2 + select * from (select 'k2' as key, '' as value from src limit 2)b union all - select 'k3' as key, ' ' as value from src limit 2 + select * from (select 'k3' as key, ' ' as value from src limit 2)c ) T POSTHOOK: type: QUERY POSTHOOK: Input: default@src Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/metadata_only_queries.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/metadata_only_queries.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/metadata_only_queries.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/metadata_only_queries.q.out Wed Feb 11 17:48:36 2015 @@ -350,7 +350,7 @@ POSTHOOK: query: select count(*), sum(1) POSTHOOK: type: QUERY POSTHOOK: Input: default@stats_tbl #### A masked pattern was here #### -9999 9999 1999.8 9999 9999 9999 9999 9999 +9999 9999 1999.8000000000002 9999 9999 9999 9999 9999 PREHOOK: query: explain select min(i), max(i), min(b), max(b), min(f), max(f), min(d), max(d) from stats_tbl PREHOOK: type: QUERY @@ -375,7 +375,7 @@ POSTHOOK: query: select min(i), max(i), POSTHOOK: type: QUERY POSTHOOK: Input: default@stats_tbl #### A masked pattern was here #### -65536 65791 4294967296 4294967551 0.009999999776482582 99.9800033569336 0.01 50.0 +65536 65791 4294967296 4294967551 0.01 99.98 0.01 50.0 PREHOOK: query: explain select count(*), sum(1), sum(0.2), count(1), count(s), count(bo), count(bin), count(si) from stats_tbl_part PREHOOK: type: QUERY @@ -400,7 +400,7 @@ POSTHOOK: query: select count(*), sum(1) POSTHOOK: type: QUERY POSTHOOK: Input: default@stats_tbl_part #### A masked pattern was here #### -9489 9489 1897.8 9489 9489 9489 9489 9489 +9489 9489 1897.8000000000002 9489 9489 9489 9489 9489 PREHOOK: query: explain select min(i), max(i), min(b), max(b), min(f), max(f), min(d), max(d) from stats_tbl_part PREHOOK: type: QUERY @@ -425,7 +425,7 @@ POSTHOOK: query: select min(i), max(i), POSTHOOK: type: QUERY POSTHOOK: Input: default@stats_tbl_part #### A masked pattern was here #### -65536 65791 4294967296 4294967551 0.009999999776482582 99.9800033569336 0.01 50.0 +65536 65791 4294967296 4294967551 0.01 99.98 0.01 50.0 PREHOOK: query: explain select count(ts) from stats_tbl_part PREHOOK: type: QUERY POSTHOOK: query: explain select count(ts) from stats_tbl_part Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/metadata_only_queries_with_filters.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/metadata_only_queries_with_filters.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/metadata_only_queries_with_filters.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/metadata_only_queries_with_filters.q.out Wed Feb 11 17:48:36 2015 @@ -166,7 +166,7 @@ POSTHOOK: query: select count(*), count( POSTHOOK: type: QUERY POSTHOOK: Input: default@stats_tbl_part #### A masked pattern was here #### -2322 2322 2322 2322 2322 2322 2322 65791 4294967296 99.9800033569336 0.03 +2322 2322 2322 2322 2322 2322 2322 65791 4294967296 99.98 0.03 PREHOOK: query: explain select count(*), count(1), sum(1), sum(2), count(s), count(bo), count(bin), count(si), max(i), min(b), max(f), min(d) from stats_tbl_part where dt > 2010 PREHOOK: type: QUERY @@ -191,7 +191,7 @@ POSTHOOK: query: select count(*), count( POSTHOOK: type: QUERY POSTHOOK: Input: default@stats_tbl_part #### A masked pattern was here #### -2219 2219 2219 4438 2219 2219 2219 2219 65791 4294967296 99.95999908447266 0.04 +2219 2219 2219 4438 2219 2219 2219 2219 65791 4294967296 99.96 0.04 PREHOOK: query: select count(*) from stats_tbl_part PREHOOK: type: QUERY PREHOOK: Input: default@stats_tbl_part Modified: hive/branches/parquet/ql/src/test/results/clientpositive/spark/optimize_nullscan.q.out URL: http://svn.apache.org/viewvc/hive/branches/parquet/ql/src/test/results/clientpositive/spark/optimize_nullscan.q.out?rev=1659014&r1=1659013&r2=1659014&view=diff ============================================================================== --- hive/branches/parquet/ql/src/test/results/clientpositive/spark/optimize_nullscan.q.out (original) +++ hive/branches/parquet/ql/src/test/results/clientpositive/spark/optimize_nullscan.q.out Wed Feb 11 17:48:36 2015 @@ -595,7 +595,7 @@ ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_SUBQUERY - TOK_UNION + TOK_UNIONALL TOK_QUERY TOK_FROM TOK_TABREF @@ -1443,7 +1443,7 @@ ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_SUBQUERY - TOK_UNION + TOK_UNIONALL TOK_QUERY TOK_FROM TOK_TABREF