drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jinfeng Ni <jinfengn...@gmail.com>
Subject Re: Drill calcite tracing issue
Date Fri, 10 Jul 2015 20:22:33 GMT
DRILL-3156 was filed to track the calcite trace issue [1].

Basically, the HepPlanner used for window function planning caused the
tracing issue. I have a prototype patch to fix this issue. I'll try to see
if I can get it ready for 1.2.0 release.

As a workaround, if you do not use window function, you may consider
skipping the HepPlanner findBestExp() call in
 DefaultSqlHandler.:convertToRel(SqlNode node)[2].


1. https://issues.apache.org/jira/browse/DRILL-3156
2.
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java#L451



On Fri, Jul 10, 2015 at 9:19 AM, George Spofford <geospofford@gmail.com>
wrote:

> (If there's a better target for an issue request, please let me know!)
>
> While trying to understand the details Calcite rule execution, I turned on
> the Calcite tracing per
> https://calcite.incubator.apache.org/docs/howto.html#tracing . At that
> point (running a query from the web UI)  I get the error
>
> Query Failed: An Error Occurred
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> AssertionError: Internal error: should never get here ...
>
>
> The query is pretty straightforward:
>
> select Person, sum(Qty1) from mongo.mine.test group by Person
>
>
>
> A simple trial with the same settings against the Apache Calcite
> example/csv doesn't show the same behavior. (Calcite query: SELECT DEPTNO,
> SUM(EMPNO) FROM emps GROUP BY DEPTNO ; )
>
> In Drill, I'm querying against a Mongo db but the code path in the
> exception trace doesn't immediately appear to be relevant for that. It
> seems to happen the very first time dumpGraph is called.
>
> The innermost cause (in all its glory) is due to the method:
>
> @Override public RelOptCost computeSelfCost(RelOptPlanner planner) {
>   // HepRelMetadataProvider is supposed to intercept this
>   // and redirect to the real rels.
>   throw Util.newInternal("should never get here");
> }
>
>
> and the trace is:
>
>           cause {
>             exception_class: "java.lang.AssertionError"
>             message: "Internal error: should never get here"
>             stack_trace {
>               class_name: "org.apache.calcite.util.Util"
>               file_name: "Util.java"
>               line_number: 775
>               method_name: "newInternal"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "org.apache.calcite.plan.hep.HepRelVertex"
>               file_name: "HepRelVertex.java"
>               line_number: 68
>               method_name: "computeSelfCost"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows"
>               file_name: "RelMdPercentageOriginalRows.java"
>               line_number: 165
>               method_name: "getNonCumulativeCost"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "..."
>               line_number: 0
>               method_name: "..."
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1"
>               file_name: "ReflectiveRelMetadataProvider.java"
>               line_number: 194
>               method_name: "invoke"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "..."
>               line_number: 0
>               method_name: "..."
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.calcite.rel.metadata.RelMetadataQuery"
>               file_name: "RelMetadataQuery.java"
>               line_number: 115
>               method_name: "getNonCumulativeCost"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows"
>               file_name: "RelMdPercentageOriginalRows.java"
>               line_number: 151
>               method_name: "getCumulativeCost"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "..."
>               line_number: 0
>               method_name: "..."
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1"
>               file_name: "ReflectiveRelMetadataProvider.java"
>               line_number: 194
>               method_name: "invoke"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "..."
>               line_number: 0
>               method_name: "..."
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.calcite.rel.metadata.RelMetadataQuery"
>               file_name: "RelMetadataQuery.java"
>               line_number: 101
>               method_name: "getCumulativeCost"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows"
>               file_name: "RelMdPercentageOriginalRows.java"
>               line_number: 154
>               method_name: "getCumulativeCost"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "..."
>               line_number: 0
>               method_name: "..."
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1"
>               file_name: "ReflectiveRelMetadataProvider.java"
>               line_number: 194
>               method_name: "invoke"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "..."
>               line_number: 0
>               method_name: "..."
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.calcite.rel.metadata.RelMetadataQuery"
>               file_name: "RelMetadataQuery.java"
>               line_number: 101
>               method_name: "getCumulativeCost"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "org.apache.calcite.plan.AbstractRelOptPlanner"
>               file_name: "AbstractRelOptPlanner.java"
>               line_number: 251
>               method_name: "getCost"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "org.apache.calcite.plan.hep.HepPlanner"
>               file_name: "HepPlanner.java"
>               line_number: 962
>               method_name: "dumpGraph"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "org.apache.calcite.plan.hep.HepPlanner"
>               file_name: "HepPlanner.java"
>               line_number: 151
>               method_name: "setRoot"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler"
>               file_name: "DefaultSqlHandler.java"
>               line_number: 449
>               method_name: "convertToRel"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler"
>               file_name: "DefaultSqlHandler.java"
>               line_number: 191
>               method_name: "validateAndConvert"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler"
>               file_name: "DefaultSqlHandler.java"
>               line_number: 157
>               method_name: "getPlan"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name:
> "org.apache.drill.exec.planner.sql.DrillSqlWorker"
>               file_name: "DrillSqlWorker.java"
>               line_number: 178
>               method_name: "getPlan"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "org.apache.drill.exec.work.foreman.Foreman"
>               file_name: "Foreman.java"
>               line_number: 903
>               method_name: "runSQL"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "org.apache.drill.exec.work.foreman.Foreman"
>               file_name: "Foreman.java"
>               line_number: 242
>               method_name: "run"
>               is_native_method: false
>             }
>             stack_trace {
>               class_name: "..."
>               line_number: 0
>               method_name: "..."
>               is_native_method: false
>             }
>           }
>
>
>
> The mongo database is simplicity itself, for repro purposes:
>
> {
> "Date" : "4/4/2014",
> "Person" : "Bob",
> "Qty1" : 45,
> "Qty2" : 52
> }
> {
> "Date" : "4/12/2014",
> "Person" : "Sarah",
> "Qty1" : 10,
> "Qty2" : 14
> }
> {
> "Date" : "4/12/2014",
> "Person" : "Bob",
> "Qty1" : 72,
> "Qty2" : 33
> }
> {
> "Date" : "5/1/2014",
> "Person" : "Agnes",
> "Qty1" : 24,
> "Qty2" : 25
> }
> {
> "Date" : "5/2/2014",
> "Person" : "Sarah",
> "Qty1" : 86,
> "Qty2" : 4
> }
> {
> "Date" : "5/6/2014",
> "Person" : "Bob",
> "Qty1" : 17,
> "Qty2" : 4
> }
>
> loaded with `mongoimport --db mine --collection test --drop --file
> c:/tmp/work1a.json`
>
> Thank you for any pointers.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message