hive-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gunt...@apache.org
Subject hive git commit: HIVE-10244: Vectorization : TPC-DS Q80 fails with java.lang.ClassCastException when hive.vectorized.execution.reduce.enabled is enabled (Matt McCline reviewed by John Pullokkaran)
Date Thu, 28 May 2015 23:18:41 GMT
Repository: hive
Updated Branches:
  refs/heads/branch-1.2 e592e1a22 -> a47bf7369


HIVE-10244: Vectorization : TPC-DS Q80 fails with java.lang.ClassCastException when hive.vectorized.execution.reduce.enabled
is enabled (Matt McCline reviewed by John Pullokkaran)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/a47bf736
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/a47bf736
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/a47bf736

Branch: refs/heads/branch-1.2
Commit: a47bf7369d7ca4e7e926c30eec8bd553664a7df4
Parents: e592e1a
Author: Gunther Hagleitner <gunther@apache.org>
Authored: Thu May 28 15:58:34 2015 -0700
Committer: Gunther Hagleitner <gunther@apache.org>
Committed: Thu May 28 16:06:24 2015 -0700

----------------------------------------------------------------------
 data/files/store_200                            |  12 +
 .../test/resources/testconfiguration.properties |   1 +
 .../hive/ql/optimizer/physical/Vectorizer.java  |   4 +
 .../clientpositive/vector_grouping_sets.q       |  63 +++++
 .../tez/vector_grouping_sets.q.out              | 266 +++++++++++++++++++
 .../clientpositive/vector_grouping_sets.q.out   | 253 ++++++++++++++++++
 6 files changed, 599 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hive/blob/a47bf736/data/files/store_200
----------------------------------------------------------------------
diff --git a/data/files/store_200 b/data/files/store_200
new file mode 100644
index 0000000..078bafd
--- /dev/null
+++ b/data/files/store_200
@@ -0,0 +1,12 @@
+1|AAAAAAAABAAAAAAA|1997-03-13||2451189|ought|245|5250760|8AM-4PM|William Ward|2|Unknown|Enough
high areas stop expectations. Elaborate, local is|Charles Bartley|1|Unknown|1|Unknown|767|Spring
|Wy|Suite 250|Midway|Williamson County|TN|31904|United States|-5|0.03|
+2|AAAAAAAACAAAAAAA|1997-03-13|2000-03-12||able|236|5285950|8AM-4PM|Scott Smith|8|Unknown|Parliamentary
candidates wait then heavy, keen mil|David Lamontagne|1|Unknown|1|Unknown|255|Sycamore |Dr.|Suite
410|Midway|Williamson County|TN|31904|United States|-5|0.03|
+3|AAAAAAAACAAAAAAA|2000-03-13|||able|236|7557959|8AM-4PM|Scott Smith|7|Unknown|Impossible,
true arms can treat constant, complete w|David Lamontagne|1|Unknown|1|Unknown|877|Park Laurel|Road|Suite
T|Midway|Williamson County|TN|31904|United States|-5|0.03|
+4|AAAAAAAAEAAAAAAA|1997-03-13|1999-03-13|2451044|ese|218|9341467|8AM-4PM|Edwin Adams|4|Unknown|Events
would achieve other, eastern hours. Mechanisms must not eat other, new org|Thomas Pollack|1|Unknown|1|Unknown|27|Lake
|Ln|Suite 260|Midway|Williamson County|TN|31904|United States|-5|0.03|
+5|AAAAAAAAEAAAAAAA|1999-03-14|2001-03-12|2450910|anti|288|9078805|8AM-4PM|Edwin Adams|8|Unknown|Events
would achieve other, eastern hours. Mechanisms must not eat other, new org|Thomas Pollack|1|Unknown|1|Unknown|27|Lee
6th|Court|Suite 80|Fairview|Williamson County|TN|35709|United States|-5|0.03|
+6|AAAAAAAAEAAAAAAA|2001-03-13|||cally|229|9026222|8AM-4PM|Edwin Adams|10|Unknown|Events would
achieve other, eastern hours. Mechanisms must not eat other, new org|Thomas Pollack|1|Unknown|1|Unknown|220|6th
|Lane|Suite 140|Midway|Williamson County|TN|31904|United States|-5|0.03|
+7|AAAAAAAAHAAAAAAA|1997-03-13|||ation|297|8954883|8AM-4PM|David Thomas|9|Unknown|Architects
coul|Thomas Benton|1|Unknown|1|Unknown|811|Lee |Circle|Suite T|Midway|Williamson County|TN|31904|United
States|-5|0.01|
+8|AAAAAAAAIAAAAAAA|1997-03-13|2000-03-12||eing|278|6995995|8AM-4PM|Brett Yates|2|Unknown|Various
bars make most. Difficult levels introduce at a boots. Buildings welcome only never el|Dean
Morrison|1|Unknown|1|Unknown|226|12th |Lane|Suite D|Fairview|Williamson County|TN|35709|United
States|-5|0.08|
+9|AAAAAAAAIAAAAAAA|2000-03-13|||eing|271|6995995|8AM-4PM|Brett Yates|2|Unknown|Formal, psychological
pounds relate reasonable, young principles. Black, |Dean Morrison|1|Unknown|1|Unknown|226|Hill
|Boulevard|Suite 190|Midway|Williamson County|TN|31904|United States|-5|0.08|
+10|AAAAAAAAKAAAAAAA|1997-03-13|1999-03-13||bar|294|9294113|8AM-4PM|Raymond Jacobs|8|Unknown|Little
expectations include yet forward meetings.|Michael Wilson|1|Unknown|1|Unknown|175|4th |Court|Suite
C|Midway|Williamson County|TN|31904|United States|-5|0.06|
+11|AAAAAAAAKAAAAAAA|1999-03-14|2001-03-12||ought|294|9294113|8AM-4PM|Raymond Jacobs|6|Unknown|Mysterious
employe|Michael Wilson|1|Unknown|1|Unknown|175|Park Green|Court|Suite 160|Midway|Williamson
County|TN|31904|United States|-5|0.11|
+12|AAAAAAAAKAAAAAAA|2001-03-13|||ought|294|5219562|8AM-12AM|Robert Thompson|6|Unknown|Events
develop i|Dustin Kelly|1|Unknown|1|Unknown|337|College |Boulevard|Suite 100|Fairview|Williamson
County|TN|31904|United States|-5|0.01|

http://git-wip-us.apache.org/repos/asf/hive/blob/a47bf736/itests/src/test/resources/testconfiguration.properties
----------------------------------------------------------------------
diff --git a/itests/src/test/resources/testconfiguration.properties b/itests/src/test/resources/testconfiguration.properties
index 376c7af..bdb96e8 100644
--- a/itests/src/test/resources/testconfiguration.properties
+++ b/itests/src/test/resources/testconfiguration.properties
@@ -220,6 +220,7 @@ minitez.query.files.shared=alter_merge_2_orc.q,\
   vector_elt.q,\
   vector_groupby_3.q,\
   vector_groupby_reduce.q,\
+  vector_grouping_sets.q,\
   vector_if_expr.q,\
   vector_inner_join.q,\
   vector_interval_1.q,\

http://git-wip-us.apache.org/repos/asf/hive/blob/a47bf736/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
index a36ee2c..bda2352 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
@@ -1119,6 +1119,10 @@ public class Vectorizer implements PhysicalPlanResolver {
       LOG.info("Grouping sets not supported in vector mode");
       return false;
     }
+    if (desc.pruneGroupingSetId()) {
+      LOG.info("Pruning grouping set id not supported in vector mode");
+      return false;
+    }
     boolean ret = validateExprNodeDesc(desc.getKeys());
     if (!ret) {
       LOG.info("Cannot vectorize groupby key expression");

http://git-wip-us.apache.org/repos/asf/hive/blob/a47bf736/ql/src/test/queries/clientpositive/vector_grouping_sets.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vector_grouping_sets.q b/ql/src/test/queries/clientpositive/vector_grouping_sets.q
new file mode 100644
index 0000000..9449ec4
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/vector_grouping_sets.q
@@ -0,0 +1,63 @@
+SET hive.vectorized.execution.enabled=true;
+set hive.fetch.task.conversion=none;
+
+-- We are not expecting either query to vectorize due to use of pruneing grouping set id,
+-- or use of GROUPING__ID virtual column.
+create table store_txt
+(
+    s_store_sk                int,
+    s_store_id                string,
+    s_rec_start_date          string,
+    s_rec_end_date            string,
+    s_closed_date_sk          int,
+    s_store_name              string,
+    s_number_employees        int,
+    s_floor_space             int,
+    s_hours                   string,
+    s_manager                 string,
+    s_market_id               int,
+    s_geography_class         string,
+    s_market_desc             string,
+    s_market_manager          string,
+    s_division_id             int,
+    s_division_name           string,
+    s_company_id              int,
+    s_company_name            string,
+    s_street_number           string,
+    s_street_name             string,
+    s_street_type             string,
+    s_suite_number            string,
+    s_city                    string,
+    s_county                  string,
+    s_state                   string,
+    s_zip                     string,
+    s_country                 string,
+    s_gmt_offset              decimal(5,2),
+    s_tax_precentage          decimal(5,2)                  
+)
+row format delimited fields terminated by '|' 
+stored as textfile;
+
+LOAD DATA LOCAL INPATH '../../data/files/store_200' OVERWRITE INTO TABLE store_txt;
+
+create table store
+stored as orc as
+select * from store_txt;
+
+explain
+select s_store_id
+ from store
+ group by s_store_id with rollup;
+
+select s_store_id
+ from store
+ group by s_store_id with rollup;
+
+explain
+select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup;
+
+select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup;
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/hive/blob/a47bf736/ql/src/test/results/clientpositive/tez/vector_grouping_sets.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/tez/vector_grouping_sets.q.out b/ql/src/test/results/clientpositive/tez/vector_grouping_sets.q.out
new file mode 100644
index 0000000..688d2ac
--- /dev/null
+++ b/ql/src/test/results/clientpositive/tez/vector_grouping_sets.q.out
@@ -0,0 +1,266 @@
+PREHOOK: query: -- We are not expecting either query to vectorize due to use of pruneing
grouping set id,
+-- or use of GROUPING__ID virtual column.
+create table store_txt
+(
+    s_store_sk                int,
+    s_store_id                string,
+    s_rec_start_date          string,
+    s_rec_end_date            string,
+    s_closed_date_sk          int,
+    s_store_name              string,
+    s_number_employees        int,
+    s_floor_space             int,
+    s_hours                   string,
+    s_manager                 string,
+    s_market_id               int,
+    s_geography_class         string,
+    s_market_desc             string,
+    s_market_manager          string,
+    s_division_id             int,
+    s_division_name           string,
+    s_company_id              int,
+    s_company_name            string,
+    s_street_number           string,
+    s_street_name             string,
+    s_street_type             string,
+    s_suite_number            string,
+    s_city                    string,
+    s_county                  string,
+    s_state                   string,
+    s_zip                     string,
+    s_country                 string,
+    s_gmt_offset              decimal(5,2),
+    s_tax_precentage          decimal(5,2)                  
+)
+row format delimited fields terminated by '|' 
+stored as textfile
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@store_txt
+POSTHOOK: query: -- We are not expecting either query to vectorize due to use of pruneing
grouping set id,
+-- or use of GROUPING__ID virtual column.
+create table store_txt
+(
+    s_store_sk                int,
+    s_store_id                string,
+    s_rec_start_date          string,
+    s_rec_end_date            string,
+    s_closed_date_sk          int,
+    s_store_name              string,
+    s_number_employees        int,
+    s_floor_space             int,
+    s_hours                   string,
+    s_manager                 string,
+    s_market_id               int,
+    s_geography_class         string,
+    s_market_desc             string,
+    s_market_manager          string,
+    s_division_id             int,
+    s_division_name           string,
+    s_company_id              int,
+    s_company_name            string,
+    s_street_number           string,
+    s_street_name             string,
+    s_street_type             string,
+    s_suite_number            string,
+    s_city                    string,
+    s_county                  string,
+    s_state                   string,
+    s_zip                     string,
+    s_country                 string,
+    s_gmt_offset              decimal(5,2),
+    s_tax_precentage          decimal(5,2)                  
+)
+row format delimited fields terminated by '|' 
+stored as textfile
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@store_txt
+PREHOOK: query: LOAD DATA LOCAL INPATH '../../data/files/store_200' OVERWRITE INTO TABLE
store_txt
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@store_txt
+POSTHOOK: query: LOAD DATA LOCAL INPATH '../../data/files/store_200' OVERWRITE INTO TABLE
store_txt
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@store_txt
+PREHOOK: query: create table store
+stored as orc as
+select * from store_txt
+PREHOOK: type: CREATETABLE_AS_SELECT
+PREHOOK: Input: default@store_txt
+PREHOOK: Output: database:default
+PREHOOK: Output: default@store
+POSTHOOK: query: create table store
+stored as orc as
+select * from store_txt
+POSTHOOK: type: CREATETABLE_AS_SELECT
+POSTHOOK: Input: default@store_txt
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@store
+PREHOOK: query: explain
+select s_store_id
+ from store
+ group by s_store_id with rollup
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select s_store_id
+ from store
+ group by s_store_id with rollup
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+  Stage-1 is a root stage
+  Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+  Stage: Stage-1
+    Tez
+      Edges:
+        Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+      Vertices:
+        Map 1 
+            Map Operator Tree:
+                TableScan
+                  alias: store
+                  Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column
stats: NONE
+                  Select Operator
+                    expressions: s_store_id (type: string)
+                    outputColumnNames: _col0
+                    Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column
stats: NONE
+                    Group By Operator
+                      keys: _col0 (type: string), '0' (type: string)
+                      mode: hash
+                      outputColumnNames: _col0, _col1
+                      Statistics: Num rows: 24 Data size: 51264 Basic stats: COMPLETE Column
stats: NONE
+                      Reduce Output Operator
+                        key expressions: _col0 (type: string), _col1 (type: string)
+                        sort order: ++
+                        Map-reduce partition columns: _col0 (type: string), _col1 (type:
string)
+                        Statistics: Num rows: 24 Data size: 51264 Basic stats: COMPLETE Column
stats: NONE
+        Reducer 2 
+            Reduce Operator Tree:
+              Group By Operator
+                keys: KEY._col0 (type: string), KEY._col1 (type: string)
+                mode: mergepartial
+                outputColumnNames: _col0
+                Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats:
NONE
+                pruneGroupingSetId: true
+                File Output Operator
+                  compressed: false
+                  Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column
stats: NONE
+                  table:
+                      input format: org.apache.hadoop.mapred.TextInputFormat
+                      output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+                      serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+  Stage: Stage-0
+    Fetch Operator
+      limit: -1
+      Processor Tree:
+        ListSink
+
+PREHOOK: query: select s_store_id
+ from store
+ group by s_store_id with rollup
+PREHOOK: type: QUERY
+PREHOOK: Input: default@store
+#### A masked pattern was here ####
+POSTHOOK: query: select s_store_id
+ from store
+ group by s_store_id with rollup
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@store
+#### A masked pattern was here ####
+NULL
+AAAAAAAABAAAAAAA
+AAAAAAAACAAAAAAA
+AAAAAAAAEAAAAAAA
+AAAAAAAAHAAAAAAA
+AAAAAAAAIAAAAAAA
+AAAAAAAAKAAAAAAA
+PREHOOK: query: explain
+select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+  Stage-1 is a root stage
+  Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+  Stage: Stage-1
+    Tez
+      Edges:
+        Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+      Vertices:
+        Map 1 
+            Map Operator Tree:
+                TableScan
+                  alias: store
+                  Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column
stats: NONE
+                  Select Operator
+                    expressions: s_store_id (type: string)
+                    outputColumnNames: _col0
+                    Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column
stats: NONE
+                    Group By Operator
+                      keys: _col0 (type: string), '0' (type: string)
+                      mode: hash
+                      outputColumnNames: _col0, _col1
+                      Statistics: Num rows: 24 Data size: 51264 Basic stats: COMPLETE Column
stats: NONE
+                      Reduce Output Operator
+                        key expressions: _col0 (type: string), _col1 (type: string)
+                        sort order: ++
+                        Map-reduce partition columns: _col0 (type: string), _col1 (type:
string)
+                        Statistics: Num rows: 24 Data size: 51264 Basic stats: COMPLETE Column
stats: NONE
+        Reducer 2 
+            Reduce Operator Tree:
+              Group By Operator
+                keys: KEY._col0 (type: string), KEY._col1 (type: string)
+                mode: mergepartial
+                outputColumnNames: _col0, _col1
+                Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats:
NONE
+                Select Operator
+                  expressions: _col0 (type: string), _col1 (type: string)
+                  outputColumnNames: _col0, _col1
+                  Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column
stats: NONE
+                  File Output Operator
+                    compressed: false
+                    Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column
stats: NONE
+                    table:
+                        input format: org.apache.hadoop.mapred.TextInputFormat
+                        output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+                        serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+            Execution mode: vectorized
+
+  Stage: Stage-0
+    Fetch Operator
+      limit: -1
+      Processor Tree:
+        ListSink
+
+PREHOOK: query: select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup
+PREHOOK: type: QUERY
+PREHOOK: Input: default@store
+#### A masked pattern was here ####
+POSTHOOK: query: select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@store
+#### A masked pattern was here ####
+NULL	0
+AAAAAAAABAAAAAAA	1
+AAAAAAAACAAAAAAA	1
+AAAAAAAAEAAAAAAA	1
+AAAAAAAAHAAAAAAA	1
+AAAAAAAAIAAAAAAA	1
+AAAAAAAAKAAAAAAA	1

http://git-wip-us.apache.org/repos/asf/hive/blob/a47bf736/ql/src/test/results/clientpositive/vector_grouping_sets.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/vector_grouping_sets.q.out b/ql/src/test/results/clientpositive/vector_grouping_sets.q.out
new file mode 100644
index 0000000..69d6187
--- /dev/null
+++ b/ql/src/test/results/clientpositive/vector_grouping_sets.q.out
@@ -0,0 +1,253 @@
+PREHOOK: query: -- We are not expecting either query to vectorize due to use of pruneing
grouping set id,
+-- or use of GROUPING__ID virtual column.
+create table store_txt
+(
+    s_store_sk                int,
+    s_store_id                string,
+    s_rec_start_date          string,
+    s_rec_end_date            string,
+    s_closed_date_sk          int,
+    s_store_name              string,
+    s_number_employees        int,
+    s_floor_space             int,
+    s_hours                   string,
+    s_manager                 string,
+    s_market_id               int,
+    s_geography_class         string,
+    s_market_desc             string,
+    s_market_manager          string,
+    s_division_id             int,
+    s_division_name           string,
+    s_company_id              int,
+    s_company_name            string,
+    s_street_number           string,
+    s_street_name             string,
+    s_street_type             string,
+    s_suite_number            string,
+    s_city                    string,
+    s_county                  string,
+    s_state                   string,
+    s_zip                     string,
+    s_country                 string,
+    s_gmt_offset              decimal(5,2),
+    s_tax_precentage          decimal(5,2)                  
+)
+row format delimited fields terminated by '|' 
+stored as textfile
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@store_txt
+POSTHOOK: query: -- We are not expecting either query to vectorize due to use of pruneing
grouping set id,
+-- or use of GROUPING__ID virtual column.
+create table store_txt
+(
+    s_store_sk                int,
+    s_store_id                string,
+    s_rec_start_date          string,
+    s_rec_end_date            string,
+    s_closed_date_sk          int,
+    s_store_name              string,
+    s_number_employees        int,
+    s_floor_space             int,
+    s_hours                   string,
+    s_manager                 string,
+    s_market_id               int,
+    s_geography_class         string,
+    s_market_desc             string,
+    s_market_manager          string,
+    s_division_id             int,
+    s_division_name           string,
+    s_company_id              int,
+    s_company_name            string,
+    s_street_number           string,
+    s_street_name             string,
+    s_street_type             string,
+    s_suite_number            string,
+    s_city                    string,
+    s_county                  string,
+    s_state                   string,
+    s_zip                     string,
+    s_country                 string,
+    s_gmt_offset              decimal(5,2),
+    s_tax_precentage          decimal(5,2)                  
+)
+row format delimited fields terminated by '|' 
+stored as textfile
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@store_txt
+PREHOOK: query: LOAD DATA LOCAL INPATH '../../data/files/store_200' OVERWRITE INTO TABLE
store_txt
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@store_txt
+POSTHOOK: query: LOAD DATA LOCAL INPATH '../../data/files/store_200' OVERWRITE INTO TABLE
store_txt
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@store_txt
+PREHOOK: query: create table store
+stored as orc as
+select * from store_txt
+PREHOOK: type: CREATETABLE_AS_SELECT
+PREHOOK: Input: default@store_txt
+PREHOOK: Output: database:default
+PREHOOK: Output: default@store
+POSTHOOK: query: create table store
+stored as orc as
+select * from store_txt
+POSTHOOK: type: CREATETABLE_AS_SELECT
+POSTHOOK: Input: default@store_txt
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@store
+PREHOOK: query: explain
+select s_store_id
+ from store
+ group by s_store_id with rollup
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select s_store_id
+ from store
+ group by s_store_id with rollup
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+  Stage-1 is a root stage
+  Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+  Stage: Stage-1
+    Map Reduce
+      Map Operator Tree:
+          TableScan
+            alias: store
+            Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats:
NONE
+            Select Operator
+              expressions: s_store_id (type: string)
+              outputColumnNames: _col0
+              Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats:
NONE
+              Group By Operator
+                keys: _col0 (type: string), '0' (type: string)
+                mode: hash
+                outputColumnNames: _col0, _col1
+                Statistics: Num rows: 24 Data size: 51264 Basic stats: COMPLETE Column stats:
NONE
+                Reduce Output Operator
+                  key expressions: _col0 (type: string), _col1 (type: string)
+                  sort order: ++
+                  Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
+                  Statistics: Num rows: 24 Data size: 51264 Basic stats: COMPLETE Column
stats: NONE
+      Reduce Operator Tree:
+        Group By Operator
+          keys: KEY._col0 (type: string), KEY._col1 (type: string)
+          mode: mergepartial
+          outputColumnNames: _col0
+          Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats: NONE
+          pruneGroupingSetId: true
+          File Output Operator
+            compressed: false
+            Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats:
NONE
+            table:
+                input format: org.apache.hadoop.mapred.TextInputFormat
+                output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+                serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+  Stage: Stage-0
+    Fetch Operator
+      limit: -1
+      Processor Tree:
+        ListSink
+
+PREHOOK: query: select s_store_id
+ from store
+ group by s_store_id with rollup
+PREHOOK: type: QUERY
+PREHOOK: Input: default@store
+#### A masked pattern was here ####
+POSTHOOK: query: select s_store_id
+ from store
+ group by s_store_id with rollup
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@store
+#### A masked pattern was here ####
+NULL
+AAAAAAAABAAAAAAA
+AAAAAAAACAAAAAAA
+AAAAAAAAEAAAAAAA
+AAAAAAAAHAAAAAAA
+AAAAAAAAIAAAAAAA
+AAAAAAAAKAAAAAAA
+PREHOOK: query: explain
+select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+  Stage-1 is a root stage
+  Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+  Stage: Stage-1
+    Map Reduce
+      Map Operator Tree:
+          TableScan
+            alias: store
+            Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats:
NONE
+            Select Operator
+              expressions: s_store_id (type: string)
+              outputColumnNames: _col0
+              Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats:
NONE
+              Group By Operator
+                keys: _col0 (type: string), '0' (type: string)
+                mode: hash
+                outputColumnNames: _col0, _col1
+                Statistics: Num rows: 24 Data size: 51264 Basic stats: COMPLETE Column stats:
NONE
+                Reduce Output Operator
+                  key expressions: _col0 (type: string), _col1 (type: string)
+                  sort order: ++
+                  Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
+                  Statistics: Num rows: 24 Data size: 51264 Basic stats: COMPLETE Column
stats: NONE
+      Reduce Operator Tree:
+        Group By Operator
+          keys: KEY._col0 (type: string), KEY._col1 (type: string)
+          mode: mergepartial
+          outputColumnNames: _col0, _col1
+          Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats: NONE
+          Select Operator
+            expressions: _col0 (type: string), _col1 (type: string)
+            outputColumnNames: _col0, _col1
+            Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats:
NONE
+            File Output Operator
+              compressed: false
+              Statistics: Num rows: 12 Data size: 25632 Basic stats: COMPLETE Column stats:
NONE
+              table:
+                  input format: org.apache.hadoop.mapred.TextInputFormat
+                  output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+                  serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+  Stage: Stage-0
+    Fetch Operator
+      limit: -1
+      Processor Tree:
+        ListSink
+
+PREHOOK: query: select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup
+PREHOOK: type: QUERY
+PREHOOK: Input: default@store
+#### A masked pattern was here ####
+POSTHOOK: query: select s_store_id, GROUPING__ID
+ from store
+ group by s_store_id with rollup
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@store
+#### A masked pattern was here ####
+NULL	0
+AAAAAAAABAAAAAAA	1
+AAAAAAAACAAAAAAA	1
+AAAAAAAAEAAAAAAA	1
+AAAAAAAAHAAAAAAA	1
+AAAAAAAAIAAAAAAA	1
+AAAAAAAAKAAAAAAA	1


Mime
View raw message