drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khurram Faraaz (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (DRILL-2796) Select keys from JSON file where <column-name> not in null results in RelOptPlanner.CannotPlanException
Date Wed, 29 Apr 2015 22:04:08 GMT

     [ https://issues.apache.org/jira/browse/DRILL-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Khurram Faraaz reopened DRILL-2796:
-----------------------------------

I tried to verify the fix for the issue, I see it fails with a different message now. This
is not a Can not Plan problem now.

{code}
0: jdbc:drill:> select * from sys.version;
+------------+----------------+-------------+-------------+------------+
| commit_id  | commit_message | commit_time | build_email | build_time |
+------------+----------------+-------------+-------------+------------+
| f5b0f4928d9c8c47c145a179c52ba3933d85c0b4 | DRILL-1502: Can't connect to mongo when requiring
auth | 29.04.2015 @ 11:22:41 EDT | Unknown     | 29.04.2015 @ 14:44:05 EDT |
+------------+----------------+-------------+-------------+------------+
{code}

explain plan failed

{code}
0: jdbc:drill:> explain plan for select key1,key2,key3,key4,key5,key6,key7,key8 from `mKeyJSN.json`
where key5 not in (null);
Query failed: SYSTEM ERROR: Unexpected exception during fragment initialization: Internal
error: Error while applying rule DrillValuesRule, args [rel#333:LogicalValues.NONE.ANY([]).[[0]](type=RecordType(ANY
ROW_VALUE),tuples=[{ null }])]


[f2cc0e03-edd3-41ca-afb7-3d927553b29b on centos-03.qa.lab:31010]
Error: exception while executing query: Failure while executing query. (state=,code=0)
{code}

I am seeing this stack trace in drillbit.log

{code}
2015-04-29 21:10:49,197 [2abeb725-9691-03df-b6c1-716a3d01ed91:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- State change requested.  PENDING --> FAILED
org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment
initialization: Internal error: Error while applying rule DrillValuesRule, args [rel#228:LogicalValues.NONE.ANY([]).[[0]](type=RecordType(ANY
ROW_VALUE),tuples=[{ null }])]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:212) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.AssertionError: Internal error: Error while applying rule DrillValuesRule,
args [rel#228:LogicalValues.NONE.ANY([]).[[0]](type=RecordType(ANY ROW_VALUE),tuples=[{ null
}])]
        at org.apache.calcite.util.Util.newInternal(Util.java:743) ~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251)
~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:795)
~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303) ~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:316) ~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:241)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:152)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:167)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:773) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:203) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        ... 3 common frames omitted
Caused by: java.lang.UnsupportedOperationException: Unable to convert the value of null and
type ANY to a Drill constant expression.
        at org.apache.drill.exec.planner.logical.DrillValuesRel.writeLiteral(DrillValuesRel.java:289)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.logical.DrillValuesRel.convertToJsonNode(DrillValuesRel.java:152)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.logical.DrillValuesRel.<init>(DrillValuesRel.java:77)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.logical.DrillValuesRule.onMatch(DrillValuesRule.java:40)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        ... 11 common frames omitted
2015-04-29 21:10:49,215 [2abeb725-9691-03df-b6c1-716a3d01ed91:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- foreman cleaning up.
2015-04-29 21:10:49,216 [2abeb725-9691-03df-b6c1-716a3d01ed91:foreman] ERROR o.a.d.c.e.DrillRuntimeException
- SYSTEM ERROR: Unexpected exception during fragment initialization: Internal error: Error
while applying rule DrillValuesRule, args [rel#228:LogicalValues.NONE.ANY([]).[[0]](type=RecordType(ANY
ROW_VALUE),tuples=[{ null }])]


[8fafcc7e-9b7f-42d9-80fa-4a8a707c4698 on centos-03.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: Unexpected exception during
fragment initialization: Internal error: Error while applying rule DrillValuesRule, args [rel#228:LogicalValues.NONE.ANY([]).[[0]](type=RecordType(ANY
ROW_VALUE),tuples=[{ null }])]


[8fafcc7e-9b7f-42d9-80fa-4a8a707c4698 on centos-03.qa.lab:31010]
        at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:465)
~[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:620)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:717)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:659)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73) [drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:661)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:762) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:212) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during
fragment initialization: Internal error: Error while applying rule DrillValuesRule, args [rel#228:LogicalValues.NONE.ANY([]).[[0]](type=RecordType(ANY
ROW_VALUE),tuples=[{ null }])]
        ... 4 common frames omitted
Caused by: java.lang.AssertionError: Internal error: Error while applying rule DrillValuesRule,
args [rel#228:LogicalValues.NONE.ANY([]).[[0]](type=RecordType(ANY ROW_VALUE),tuples=[{ null
}])]
        at org.apache.calcite.util.Util.newInternal(Util.java:743) ~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251)
~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:795)
~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303) ~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:316) ~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:241)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:152)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:167)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:773) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:203) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        ... 3 common frames omitted
Caused by: java.lang.UnsupportedOperationException: Unable to convert the value of null and
type ANY to a Drill constant expression.
        at org.apache.drill.exec.planner.logical.DrillValuesRel.writeLiteral(DrillValuesRel.java:289)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.logical.DrillValuesRel.convertToJsonNode(DrillValuesRel.java:152)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.logical.DrillValuesRel.<init>(DrillValuesRel.java:77)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.planner.logical.DrillValuesRule.onMatch(DrillValuesRule.java:40)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
~[calcite-core-1.1.0-drill-r2.jar:1.1.0-drill-r2]
        ... 11 common frames omitted
{code}

> Select keys from JSON file where <column-name> not in null results in RelOptPlanner.CannotPlanException
> -------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-2796
>                 URL: https://issues.apache.org/jira/browse/DRILL-2796
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 0.9.0
>         Environment:  9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580: Exit early
from HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53 EDT
>            Reporter: Khurram Faraaz
>            Assignee: Aman Sinha
>             Fix For: 1.0.0
>
>
> Query that has, <column-name> not in (null) in its predicate, results in RelOptPlanner.CannotPlanException.
Tests were run on 4 node cluster and on CentOS.
> Data is being selected from a JSON data file.
> {code}
> 0: jdbc:drill:> select * from `mKeyJSN.json`;
> +------------+------------+------------+------------+------------+------------+------------+------------+
> |    key1    |    key2    |    key3    |    key4    |    key5    |    key6    |    key7
   |    key8    |
> +------------+------------+------------+------------+------------+------------+------------+------------+
> | 1234       | null       | null       | null       | null       | null       | null
      | null       |
> | null       | 1245685    | null       | null       | null       | null       | null
      | null       |
> | null       | null       | hello world! | null       | null       | null       | null
      | null       |
> | null       | null       | null       | true       | null       | null       | null
      | null       |
> | null       | null       | null       | null       | 2000-03-10 | null       | null
      | null       |
> | null       | null       | null       | null       | null       | 2012-01-21 15:19:12.123
| null       | null       |
> | null       | null       | null       | null       | null       | null       | 21:34:32.321
| null       |
> | null       | null       | null       | null       | null       | null       | null
      | 9789.99    |
> +------------+------------+------------+------------+------------+------------+------------+------------+
> 8 rows selected (0.1 seconds)
> store.json.all_text_mode was set to false
> 0: jdbc:drill:> select * from sys.options where name like '%json.all_text_mode%';
> +------------+------------+------------+------------+------------+------------+------------+
> |    name    |    kind    |    type    |  num_val   | string_val |  bool_val  | float_val
 |
> +------------+------------+------------+------------+------------+------------+------------+
> | store.json.all_text_mode | BOOLEAN    | SYSTEM     | null       | null       | false
     | null       |
> +------------+------------+------------+------------+------------+------------+------------+
> 1 row selected (0.135 seconds)
> Failing query
> 0: jdbc:drill:> select key1,key2,key3,key4,key5,key6,key7,key8 from `mKeyJSN.json`
where key5 not in (null);
> Query failed: RelOptPlanner.CannotPlanException: Node [rel#14924:Subset#7.LOGICAL.ANY([]).[]]
could not be implemented; planner state:
> Root: rel#14924:Subset#7.LOGICAL.ANY([]).[]
> Original rel:
> AbstractConverter(subset=[rel#14924:Subset#7.LOGICAL.ANY([]).[]], convention=[LOGICAL],
DrillDistributionTraitDef=[ANY([])], sort=[[]]): rowcount = 1.7976931348623157E308, cumulative
cost = {inf}, id = 14925
>   ProjectRel(subset=[rel#14923:Subset#7.NONE.ANY([]).[]], key1=[$2], key2=[$3], key3=[$4],
key4=[$5], key5=[$1], key6=[$6], key7=[$7], key8=[$8]): rowcount = 1.7976931348623157E308,
cumulative cost = {1.7976931348623157E308 rows, Infinity cpu, 0.0 io, 0.0 network, 0.0 memory},
id = 14922
>     FilterRel(subset=[rel#14921:Subset#6.NONE.ANY([]).[]], condition=[AND(NOT(IS TRUE($11)),
IS NOT NULL($9))]): rowcount = 4.0448095534402104E307, cumulative cost = {4.0448095534402104E307
rows, 1.7976931348623157E308 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 14920
>       JoinRel(subset=[rel#14919:Subset#5.NONE.ANY([]).[]], condition=[=($9, $10)], joinType=[left]):
rowcount = 1.7976931348623157E308, cumulative cost = {1.7976931348623157E308 rows, 0.0 cpu,
0.0 io, 0.0 network, 0.0 memory}, id = 14918
>         ProjectRel(subset=[rel#14912:Subset#1.NONE.ANY([]).[]], $f0=[$0], $f1=[$1], $f2=[$2],
$f3=[$3], $f4=[$4], $f5=[$5], $f6=[$6], $f7=[$7], $f8=[$8], $f9=[$1]): rowcount = 100.0, cumulative
cost = {100.0 rows, 1000.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 14911
>           EnumerableTableAccessRel(subset=[rel#14910:Subset#0.ENUMERABLE.ANY([]).[]],
table=[[dfs, tmp, mKeyJSN.json]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0
cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 14888
>         AggregateRel(subset=[rel#14917:Subset#4.NONE.ANY([]).[]], group=[{0}], agg#0=[MIN($1)]):
rowcount = 1.7976931348623158E307, cumulative cost = {1.7976931348623158E307 rows, 0.0 cpu,
0.0 io, 0.0 network, 0.0 memory}, id = 14916
>           ProjectRel(subset=[rel#14915:Subset#3.NONE.ANY([]).[]], $f0=[$0], $f1=[true]):
rowcount = 1.7976931348623157E308, cumulative cost = {1.7976931348623157E308 rows, Infinity
cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 14914
>             ValuesRel(subset=[rel#14913:Subset#2.NONE.ANY([]).[]], tuples=[[{ null }]]):
rowcount = 1.0, cumulative cost = {1.0 rows, 1.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id
= 14890
> content from JSON data file
> {"key1":1234}
> {"key2":1245685}
> {"key3":"hello world!"}
> {"key4":true}
> {"key5":"2000-03-10"}
> {"key6":"2012-01-21 15:19:12.123"}
> {"key7":"21:34:32.321"}
> {"key8":9789.99}
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message