drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-1200) mondrian2580.q - cause schema change exception
Date Sun, 04 Jan 2015 21:29:49 GMT

     [ https://issues.apache.org/jira/browse/DRILL-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jacques Nadeau updated DRILL-1200:
----------------------------------
    Priority: Minor  (was: Major)

> mondrian2580.q - cause schema change exception
> ----------------------------------------------
>
>                 Key: DRILL-1200
>                 URL: https://issues.apache.org/jira/browse/DRILL-1200
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>            Reporter: Chun Chang
>            Priority: Minor
>             Fix For: Future
>
>
> #Mon Jul 21 10:24:21 PDT 2014
> git.commit.id.abbrev=e5c2da0
> The following mondrian query (query2580.q) caused schema change exception:
> 0: jdbc:drill:schema=dfs> select time_by_day.the_year as c0, sum(sales_fact_1997.unit_sales)
as m0, sum(sales_fact_1997.store_cost) as m1, sum(sales_fact_1997.store_sales) as m2, count(sales_fact_1997.product_id)
as m3, count(distinct sales_fact_1997.customer_id) as m4, sum((case when sales_fact_1997.promotion_id
= 0 then 0 else sales_fact_1997.store_sales end)) as m5 from time_by_day as time_by_day, sales_fact_1997
as sales_fact_1997 where sales_fact_1997.time_id = time_by_day.time_id and time_by_day.the_year
= 1997 group by time_by_day.the_year;
> Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while running query.[error_id:
"b9fa1177-505a-4ae1-9d44-52b1d40f9a92"
> endpoint {
>   address: "qa-node120.qa.lab"
>   user_port: 31010
>   control_port: 31011
>   data_port: 31012
> }
> error_type: 0
> message: "Failure while running fragment. < IllegalStateException:[ You tried to do
a batch data read operation when you were in a state of STOP.  You can only do this type of
operation when you are in a state of OK or OK_NEW_SCHEMA. ]"
> ]
> Error: exception while executing query (state=,code=0)
> physical plan:
> 14:31:16.198 [19aa0bca-d59e-4061-962d-b3fabb0f24b7:foreman] DEBUG o.a.d.e.p.s.h.DefaultSqlHandler
- Drill Physical :
> 00-00    Screen: rowcount = 1.0, cumulative cost = {874658.1 rows, 3052726.1 cpu, 0.0
io, 0.0 network}, id = 3109
> 00-01      Project(c0=[$0], m0=[$1], m1=[$2], m2=[$3], m3=[$4], m4=[$5], m5=[$6]): rowcount
= 1.0, cumulative cost = {874658.0 rows, 3052726.0 cpu, 0.0 io, 0.0 network}, id = 3108
> 00-02        Project(c0=[$0], m0=[CASE(=($2, 0), null, $1)], m1=[CASE(=($4, 0), null,
$3)], m2=[CASE(=($6, 0), null, $5)], m3=[$7], m4=[$11], m5=[CASE(=($9, 0), null, $8)]): rowcount
= 1.0, cumulative cost = {874657.0 rows, 3052698.0 cpu, 0.0 io, 0.0 network}, id = 3107
> 00-03          HashJoin(condition=[IS NOT DISTINCT FROM($0, $10)], joinType=[inner]):
rowcount = 1.0, cumulative cost = {874656.0 rows, 3052670.0 cpu, 0.0 io, 0.0 network}, id
= 3106
> 00-05            HashAgg(group=[{0}], agg#0=[$SUM0($1)], agg#1=[COUNT($1)], agg#2=[$SUM0($2)],
agg#3=[COUNT($2)], agg#4=[$SUM0($3)], agg#5=[COUNT($3)], m3=[COUNT($4)], agg#7=[$SUM0($6)],
agg#8=[COUNT($6)]): rowcount = 1.0, cumulative cost = {437326.0 rows, 1743481.5 cpu, 0.0 io,
0.0 network}, id = 3093
> 00-07              Project(c0=[$0], unit_sales=[$3], store_cost=[$4], store_sales=[$5],
product_id=[$6], customer_id=[$7], $f6=[CASE(=(CAST($8):INTEGER, 0), CAST(0):ANY, $5)]): rowcount
= 1.0, cumulative cost = {437325.0 rows, 1743365.5 cpu, 0.0 io, 0.0 network}, id = 3092
> 00-09                HashJoin(condition=[=($2, $1)], joinType=[inner]): rowcount = 1.0,
cumulative cost = {437324.0 rows, 1743337.5 cpu, 0.0 io, 0.0 network}, id = 3091
> 00-12                  SelectionVectorRemover: rowcount = 109.5, cumulative cost = {3029.5
rows, 5227.5 cpu, 0.0 io, 0.0 network}, id = 3087
> 00-15                    Filter(condition=[=(CAST($0):INTEGER, 1997)]): rowcount = 109.5,
cumulative cost = {2920.0 rows, 5118.0 cpu, 0.0 io, 0.0 network}, id = 3086
> 00-19                      Project(the_year=[$1], time_id=[$0]): rowcount = 730.0, cumulative
cost = {2190.0 rows, 2198.0 cpu, 0.0 io, 0.0 network}, id = 3085
> 00-23                        ProducerConsumer: rowcount = 730.0, cumulative cost = {1460.0
rows, 2190.0 cpu, 0.0 io, 0.0 network}, id = 3084
> 00-26                          Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
[path=maprfs:/user/root/mondrian/time_by_day]], selectionRoot=/user/root/mondrian/time_by_day,
columns=[SchemaPath [`the_year`], SchemaPath [`time_id`]]]]): rowcount = 730.0, cumulative
cost = {730.0 rows, 1460.0 cpu, 0.0 io, 0.0 network}, id = 3058
> 00-11                  Project(time_id0=[$0], unit_sales=[$1], store_cost=[$2], store_sales=[$3],
product_id=[$4], customer_id=[$5], promotion_id=[$6]): rowcount = 86837.0, cumulative cost
= {347348.0 rows, 694752.0 cpu, 0.0 io, 0.0 network}, id = 3090
> 00-14                    Project(time_id=[$4], unit_sales=[$1], store_cost=[$5], store_sales=[$6],
product_id=[$2], customer_id=[$0], promotion_id=[$3]): rowcount = 86837.0, cumulative cost
= {260511.0 rows, 694724.0 cpu, 0.0 io, 0.0 network}, id = 3089
> 00-18                      ProducerConsumer: rowcount = 86837.0, cumulative cost = {173674.0
rows, 694696.0 cpu, 0.0 io, 0.0 network}, id = 3088
> 00-22                        Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
[path=maprfs:/user/root/mondrian/sales_fact_1997]], selectionRoot=/user/root/mondrian/sales_fact_1997,
columns=[SchemaPath [`time_id`], SchemaPath [`unit_sales`], SchemaPath [`store_cost`], SchemaPath
[`store_sales`], SchemaPath [`product_id`], SchemaPath [`customer_id`], SchemaPath [`promotion_id`]]]]):
rowcount = 86837.0, cumulative cost = {86837.0 rows, 607859.0 cpu, 0.0 io, 0.0 network}, id
= 3062
> 00-04            Project(c00=[$0], m4=[$1]): rowcount = 1.0, cumulative cost = {437328.0
rows, 1309164.5 cpu, 0.0 io, 0.0 network}, id = 3105
> 00-06              HashAgg(group=[{0}], m4=[COUNT($1)]): rowcount = 1.0, cumulative cost
= {437327.0 rows, 1309156.5 cpu, 0.0 io, 0.0 network}, id = 3104
> 00-08                HashAgg(group=[{0, 1}]): rowcount = 1.0, cumulative cost = {437326.0
rows, 1309136.5 cpu, 0.0 io, 0.0 network}, id = 3103
> 00-10                  Project(c0=[$0], customer_id=[$3]): rowcount = 1.0, cumulative
cost = {437325.0 rows, 1309120.5 cpu, 0.0 io, 0.0 network}, id = 3102
> 00-13                    HashJoin(condition=[=($2, $1)], joinType=[inner]): rowcount
= 1.0, cumulative cost = {437324.0 rows, 1309112.5 cpu, 0.0 io, 0.0 network}, id = 3101
> 00-17                      SelectionVectorRemover: rowcount = 109.5, cumulative cost
= {3029.5 rows, 5227.5 cpu, 0.0 io, 0.0 network}, id = 3097
> 00-21                        Filter(condition=[=(CAST($0):INTEGER, 1997)]): rowcount
= 109.5, cumulative cost = {2920.0 rows, 5118.0 cpu, 0.0 io, 0.0 network}, id = 3096
> 00-25                          Project(the_year=[$1], time_id=[$0]): rowcount = 730.0,
cumulative cost = {2190.0 rows, 2198.0 cpu, 0.0 io, 0.0 network}, id = 3095
> 00-28                            ProducerConsumer: rowcount = 730.0, cumulative cost
= {1460.0 rows, 2190.0 cpu, 0.0 io, 0.0 network}, id = 3094
> 00-29                              Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
[path=maprfs:/user/root/mondrian/time_by_day]], selectionRoot=/user/root/mondrian/time_by_day,
columns=[SchemaPath [`the_year`], SchemaPath [`time_id`]]]]): rowcount = 730.0, cumulative
cost = {730.0 rows, 1460.0 cpu, 0.0 io, 0.0 network}, id = 3068
> 00-16                      Project(time_id0=[$0], customer_id=[$1]): rowcount = 86837.0,
cumulative cost = {347348.0 rows, 260527.0 cpu, 0.0 io, 0.0 network}, id = 3100
> 00-20                        Project(time_id=[$1], customer_id=[$0]): rowcount = 86837.0,
cumulative cost = {260511.0 rows, 260519.0 cpu, 0.0 io, 0.0 network}, id = 3099
> 00-24                          ProducerConsumer: rowcount = 86837.0, cumulative cost
= {173674.0 rows, 260511.0 cpu, 0.0 io, 0.0 network}, id = 3098
> 00-27                            Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
[path=maprfs:/user/root/mondrian/sales_fact_1997]], selectionRoot=/user/root/mondrian/sales_fact_1997,
columns=[SchemaPath [`time_id`], SchemaPath [`customer_id`]]]]): rowcount = 86837.0, cumulative
cost = {86837.0 rows, 173674.0 cpu, 0.0 io, 0.0 network}, id = 3072
> 14:31:16.352 [19aa0bca-d59e-4061-962d-b3fabb0f24b7:foreman] DEBUG o.a.d.e.p.s.h.DefaultSqlHandler
- Drill Plan :
> {
>   "head" : {
>     "version" : 1,
>     "generator" : {
>       "type" : "DefaultSqlHandler",
>       "info" : ""
>     },
>     "type" : "APACHE_DRILL_PHYSICAL",
>     "options" : [ ],
>     "queue" : 0,
>     "resultMode" : "EXEC"
>   },
>   "graph" : [ {
>     "pop" : "parquet-scan",
>     "@id" : 27,
>     "entries" : [ {
>       "path" : "maprfs:/user/root/mondrian/sales_fact_1997"
>     } ],
>     "storage" : {
>       "type" : "file",
>       "enabled" : true,
>       "connection" : "maprfs:///",
>       "workspaces" : {
>         "default" : {
>           "location" : "/user/root/mondrian/",
>           "writable" : false,
>           "storageformat" : null
>         },
>         "home" : {
>           "location" : "/",
>           "writable" : false,
>           "storageformat" : null
>         },
>         "root" : {
>           "location" : "/",
>           "writable" : false,
>           "storageformat" : null
>         },
>         "tmp" : {
>           "location" : "/tmp",
>           "writable" : true,
>           "storageformat" : "csv"
>         }
>       },
>       "formats" : {
>         "psv" : {
>           "type" : "text",
>           "extensions" : [ "tbl" ],
>           "delimiter" : "|"
>         },
>         "csv" : {
>           "type" : "text",
>           "extensions" : [ "csv" ],
>           "delimiter" : ","
>         },
>         "tsv" : {
>           "type" : "text",
>           "extensions" : [ "tsv" ],
>           "delimiter" : "\t"
>         },
>         "parquet" : {
>           "type" : "parquet"
>         },
>         "json" : {
>           "type" : "json"
>         }
>       }
>     },
>     "format" : {
>       "type" : "parquet"
>     },
>     "columns" : [ "`time_id`", "`customer_id`" ],
>     "selectionRoot" : "/user/root/mondrian/sales_fact_1997",
>     "cost" : 86837.0
>   }, {
>     "pop" : "producer-consumer",
>     "@id" : 24,
>     "child" : 27,
>     "size" : 10,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 86837.0
>   }, {
>     "pop" : "project",
>     "@id" : 20,
>     "exprs" : [ {
>       "ref" : "`time_id`",
>       "expr" : "`time_id`"
>     }, {
>       "ref" : "`customer_id`",
>       "expr" : "`customer_id`"
>     } ],
>     "child" : 24,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 86837.0
>   }, {
>     "pop" : "project",
>     "@id" : 16,
>     "exprs" : [ {
>       "ref" : "`time_id0`",
>       "expr" : "`time_id`"
>     }, {
>       "ref" : "`customer_id`",
>       "expr" : "`customer_id`"
>     } ],
>     "child" : 20,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 86837.0
>   }, {
>     "pop" : "parquet-scan",
>     "@id" : 22,
>     "entries" : [ {
>       "path" : "maprfs:/user/root/mondrian/sales_fact_1997"
>     } ],
>     "storage" : {
>       "type" : "file",
>       "enabled" : true,
>       "connection" : "maprfs:///",
>       "workspaces" : {
>         "default" : {
>           "location" : "/user/root/mondrian/",
>           "writable" : false,
>           "storageformat" : null
>         },
>         "home" : {
>           "location" : "/",
>           "writable" : false,
>           "storageformat" : null
>         },
>         "root" : {
>           "location" : "/",
>           "writable" : false,
>           "storageformat" : null
>         },
>         "tmp" : {
>           "location" : "/tmp",
>           "writable" : true,
>           "storageformat" : "csv"
>         }
>       },
>       "formats" : {
>         "psv" : {
>           "type" : "text",
>           "extensions" : [ "tbl" ],
>           "delimiter" : "|"
>         },
>         "csv" : {
>           "type" : "text",
>           "extensions" : [ "csv" ],
>           "delimiter" : ","
>         },
>         "tsv" : {
>           "type" : "text",
>           "extensions" : [ "tsv" ],
>           "delimiter" : "\t"
>         },
>         "parquet" : {
>           "type" : "parquet"
>         },
>         "json" : {
>           "type" : "json"
>         }
>       }
>     },
>     "format" : {
>       "type" : "parquet"
>     },
>     "columns" : [ "`time_id`", "`unit_sales`", "`store_cost`", "`store_sales`", "`product_id`",
"`customer_id`", "`promotion_id`" ],
>     "selectionRoot" : "/user/root/mondrian/sales_fact_1997",
>     "cost" : 86837.0
>   }, {
>     "pop" : "producer-consumer",
>     "@id" : 18,
>     "child" : 22,
>     "size" : 10,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 86837.0
>   }, {
>     "pop" : "project",
>     "@id" : 14,
>     "exprs" : [ {
>       "ref" : "`time_id`",
>       "expr" : "`time_id`"
>     }, {
>       "ref" : "`unit_sales`",
>       "expr" : "`unit_sales`"
>     }, {
>       "ref" : "`store_cost`",
>       "expr" : "`store_cost`"
>     }, {
>       "ref" : "`store_sales`",
>       "expr" : "`store_sales`"
>     }, {
>       "ref" : "`product_id`",
>       "expr" : "`product_id`"
>     }, {
>       "ref" : "`customer_id`",
>       "expr" : "`customer_id`"
>     }, {
>       "ref" : "`promotion_id`",
>       "expr" : "`promotion_id`"
>     } ],
>     "child" : 18,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 86837.0
>   }, {
>     "pop" : "project",
>     "@id" : 11,
>     "exprs" : [ {
>       "ref" : "`time_id0`",
>       "expr" : "`time_id`"
>     }, {
>       "ref" : "`unit_sales`",
>       "expr" : "`unit_sales`"
>     }, {
>       "ref" : "`store_cost`",
>       "expr" : "`store_cost`"
>     }, {
>       "ref" : "`store_sales`",
>       "expr" : "`store_sales`"
>     }, {
>       "ref" : "`product_id`",
>       "expr" : "`product_id`"
>     }, {
>       "ref" : "`customer_id`",
>       "expr" : "`customer_id`"
>     }, {
>       "ref" : "`promotion_id`",
>       "expr" : "`promotion_id`"
>     } ],
>     "child" : 14,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 86837.0
>   }, {
>     "pop" : "parquet-scan",
>     "@id" : 26,
>     "entries" : [ {
>       "path" : "maprfs:/user/root/mondrian/time_by_day"
>     } ],
>     "storage" : {
>       "type" : "file",
>       "enabled" : true,
>       "connection" : "maprfs:///",
>       "workspaces" : {
>         "default" : {
>           "location" : "/user/root/mondrian/",
>           "writable" : false,
>           "storageformat" : null
>         },
>         "home" : {
>           "location" : "/",
>           "writable" : false,
>           "storageformat" : null
>         },
>         "root" : {
>           "location" : "/",
>           "writable" : false,
>           "storageformat" : null
>         },
>         "tmp" : {
>           "location" : "/tmp",
>           "writable" : true,
>           "storageformat" : "csv"
>         }
>       },
>       "formats" : {
>         "psv" : {
>           "type" : "text",
>           "extensions" : [ "tbl" ],
>           "delimiter" : "|"
>         },
>         "csv" : {
>           "type" : "text",
>           "extensions" : [ "csv" ],
>           "delimiter" : ","
>         },
>         "tsv" : {
>           "type" : "text",
>           "extensions" : [ "tsv" ],
>           "delimiter" : "\t"
>         },
>         "parquet" : {
>           "type" : "parquet"
>         },
>         "json" : {
>           "type" : "json"
>         }
>       }
>     },
>     "format" : {
>       "type" : "parquet"
>     },
>     "columns" : [ "`the_year`", "`time_id`" ],
>     "selectionRoot" : "/user/root/mondrian/time_by_day",
>     "cost" : 730.0
>   }, {
>     "pop" : "producer-consumer",
>     "@id" : 23,
>     "child" : 26,
>     "size" : 10,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 730.0
>   }, {
>     "pop" : "project",
>     "@id" : 19,
>     "exprs" : [ {
>       "ref" : "`the_year`",
>       "expr" : "`the_year`"
>     }, {
>       "ref" : "`time_id`",
>       "expr" : "`time_id`"
>     } ],
>     "child" : 23,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 730.0
>   }, {
>     "pop" : "filter",
>     "@id" : 15,
>     "child" : 19,
>     "expr" : "equal(cast( (`the_year` ) as INT ), 1997) ",
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 109.5
>   }, {
>     "pop" : "selection-vector-remover",
>     "@id" : 12,
>     "child" : 15,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 109.5
>   }, {
>     "pop" : "hash-join",
>     "@id" : 9,
>     "left" : 12,
>     "right" : 11,
>     "conditions" : [ {
>       "relationship" : "==",
>       "left" : "`time_id`",
>       "right" : "`time_id0`"
>     } ],
>     "joinType" : "INNER",
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 0.0
>   }, {
>     "pop" : "project",
>     "@id" : 7,
>     "exprs" : [ {
>       "ref" : "`c0`",
>       "expr" : "`the_year`"
>     }, {
>       "ref" : "`unit_sales`",
>       "expr" : "`unit_sales`"
>     }, {
>       "ref" : "`store_cost`",
>       "expr" : "`store_cost`"
>     }, {
>       "ref" : "`store_sales`",
>       "expr" : "`store_sales`"
>     }, {
>       "ref" : "`product_id`",
>       "expr" : "`product_id`"
>     }, {
>       "ref" : "`customer_id`",
>       "expr" : "`customer_id`"
>     }, {
>       "ref" : "`$f6`",
>       "expr" : " ( if (equal(cast( (`promotion_id` ) as INT ), 0)  ) then (0 )  else
(`store_sales` )  end  ) "
>     } ],
>     "child" : 9,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 1.0
>   }, {
>     "pop" : "hash-aggregate",
>     "@id" : 5,
>     "child" : 7,
>     "cardinality" : 1.0,
>     "initialAllocation" : 1000000,
>     "maxAllocation" : 10000000000,
>     "cost" : 0.5,
>     "groupByExprs" : [ {
>       "ref" : "`c0`",
>       "expr" : "`c0`"
>     } ],
>     "aggrExprs" : [ {
>       "ref" : "`$f1`",
>       "expr" : "$sum0(`unit_sales`) "
>     }, {
>       "ref" : "`$f2`",
>       "expr" : "count(`unit_sales`) "
>     }, {
>       "ref" : "`$f3`",
>       "expr" : "$sum0(`store_cost`) "
>     }, {
>       "ref" : "`$f4`",
>       "expr" : "count(`store_cost`) "
>     }, {
>       "ref" : "`$f5`",
>       "expr" : "$sum0(`store_sales`) "
>     }, {
>       "ref" : "`$f6`",
>       "expr" : "count(`store_sales`) "
>     }, {
>       "ref" : "`m3`",
>       "expr" : "count(`product_id`) "
>     }, {
>       "ref" : "`$f8`",
>       "expr" : "$sum0(`$f6`) "
>     }, {
>       "ref" : "`$f9`",
>       "expr" : "count(`$f6`) "
>     } ]
>   }, {
>     "pop" : "parquet-scan",
>     "@id" : 29,
>     "entries" : [ {
>       "path" : "maprfs:/user/root/mondrian/time_by_day"
>     } ],
>     "storage" : {
>       "type" : "file",
>       "enabled" : true,
>       "connection" : "maprfs:///",
>       "workspaces" : {
>         "default" : {
>           "location" : "/user/root/mondrian/",
>           "writable" : false,
> ....
> This is too long. Will regenerate the plan if needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message