drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Chen <tnac...@gmail.com>
Subject Re: Drill calcite tracing issue
Date Fri, 10 Jul 2015 20:29:08 GMT
I remember introducing the HepPlanner that when I worked on window
functions, probably a good idea to comment on why we actually need
that since I can't recall the exact reasons now.

Tim

On Fri, Jul 10, 2015 at 1:22 PM, Jinfeng Ni <jinfengni99@gmail.com> wrote:
> DRILL-3156 was filed to track the calcite trace issue [1].
>
> Basically, the HepPlanner used for window function planning caused the
> tracing issue. I have a prototype patch to fix this issue. I'll try to see
> if I can get it ready for 1.2.0 release.
>
> As a workaround, if you do not use window function, you may consider
> skipping the HepPlanner findBestExp() call in
>  DefaultSqlHandler.:convertToRel(SqlNode node)[2].
>
>
> 1. https://issues.apache.org/jira/browse/DRILL-3156
> 2.
> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java#L451
>
>
>
> On Fri, Jul 10, 2015 at 9:19 AM, George Spofford <geospofford@gmail.com>
> wrote:
>
>> (If there's a better target for an issue request, please let me know!)
>>
>> While trying to understand the details Calcite rule execution, I turned on
>> the Calcite tracing per
>> https://calcite.incubator.apache.org/docs/howto.html#tracing . At that
>> point (running a query from the web UI)  I get the error
>>
>> Query Failed: An Error Occurred
>> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
>> AssertionError: Internal error: should never get here ...
>>
>>
>> The query is pretty straightforward:
>>
>> select Person, sum(Qty1) from mongo.mine.test group by Person
>>
>>
>>
>> A simple trial with the same settings against the Apache Calcite
>> example/csv doesn't show the same behavior. (Calcite query: SELECT DEPTNO,
>> SUM(EMPNO) FROM emps GROUP BY DEPTNO ; )
>>
>> In Drill, I'm querying against a Mongo db but the code path in the
>> exception trace doesn't immediately appear to be relevant for that. It
>> seems to happen the very first time dumpGraph is called.
>>
>> The innermost cause (in all its glory) is due to the method:
>>
>> @Override public RelOptCost computeSelfCost(RelOptPlanner planner) {
>>   // HepRelMetadataProvider is supposed to intercept this
>>   // and redirect to the real rels.
>>   throw Util.newInternal("should never get here");
>> }
>>
>>
>> and the trace is:
>>
>>           cause {
>>             exception_class: "java.lang.AssertionError"
>>             message: "Internal error: should never get here"
>>             stack_trace {
>>               class_name: "org.apache.calcite.util.Util"
>>               file_name: "Util.java"
>>               line_number: 775
>>               method_name: "newInternal"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "org.apache.calcite.plan.hep.HepRelVertex"
>>               file_name: "HepRelVertex.java"
>>               line_number: 68
>>               method_name: "computeSelfCost"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows"
>>               file_name: "RelMdPercentageOriginalRows.java"
>>               line_number: 165
>>               method_name: "getNonCumulativeCost"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "..."
>>               line_number: 0
>>               method_name: "..."
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1"
>>               file_name: "ReflectiveRelMetadataProvider.java"
>>               line_number: 194
>>               method_name: "invoke"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "..."
>>               line_number: 0
>>               method_name: "..."
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.calcite.rel.metadata.RelMetadataQuery"
>>               file_name: "RelMetadataQuery.java"
>>               line_number: 115
>>               method_name: "getNonCumulativeCost"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows"
>>               file_name: "RelMdPercentageOriginalRows.java"
>>               line_number: 151
>>               method_name: "getCumulativeCost"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "..."
>>               line_number: 0
>>               method_name: "..."
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1"
>>               file_name: "ReflectiveRelMetadataProvider.java"
>>               line_number: 194
>>               method_name: "invoke"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "..."
>>               line_number: 0
>>               method_name: "..."
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.calcite.rel.metadata.RelMetadataQuery"
>>               file_name: "RelMetadataQuery.java"
>>               line_number: 101
>>               method_name: "getCumulativeCost"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows"
>>               file_name: "RelMdPercentageOriginalRows.java"
>>               line_number: 154
>>               method_name: "getCumulativeCost"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "..."
>>               line_number: 0
>>               method_name: "..."
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$2$1"
>>               file_name: "ReflectiveRelMetadataProvider.java"
>>               line_number: 194
>>               method_name: "invoke"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "..."
>>               line_number: 0
>>               method_name: "..."
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.calcite.rel.metadata.RelMetadataQuery"
>>               file_name: "RelMetadataQuery.java"
>>               line_number: 101
>>               method_name: "getCumulativeCost"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "org.apache.calcite.plan.AbstractRelOptPlanner"
>>               file_name: "AbstractRelOptPlanner.java"
>>               line_number: 251
>>               method_name: "getCost"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "org.apache.calcite.plan.hep.HepPlanner"
>>               file_name: "HepPlanner.java"
>>               line_number: 962
>>               method_name: "dumpGraph"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "org.apache.calcite.plan.hep.HepPlanner"
>>               file_name: "HepPlanner.java"
>>               line_number: 151
>>               method_name: "setRoot"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler"
>>               file_name: "DefaultSqlHandler.java"
>>               line_number: 449
>>               method_name: "convertToRel"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler"
>>               file_name: "DefaultSqlHandler.java"
>>               line_number: 191
>>               method_name: "validateAndConvert"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler"
>>               file_name: "DefaultSqlHandler.java"
>>               line_number: 157
>>               method_name: "getPlan"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name:
>> "org.apache.drill.exec.planner.sql.DrillSqlWorker"
>>               file_name: "DrillSqlWorker.java"
>>               line_number: 178
>>               method_name: "getPlan"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "org.apache.drill.exec.work.foreman.Foreman"
>>               file_name: "Foreman.java"
>>               line_number: 903
>>               method_name: "runSQL"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "org.apache.drill.exec.work.foreman.Foreman"
>>               file_name: "Foreman.java"
>>               line_number: 242
>>               method_name: "run"
>>               is_native_method: false
>>             }
>>             stack_trace {
>>               class_name: "..."
>>               line_number: 0
>>               method_name: "..."
>>               is_native_method: false
>>             }
>>           }
>>
>>
>>
>> The mongo database is simplicity itself, for repro purposes:
>>
>> {
>> "Date" : "4/4/2014",
>> "Person" : "Bob",
>> "Qty1" : 45,
>> "Qty2" : 52
>> }
>> {
>> "Date" : "4/12/2014",
>> "Person" : "Sarah",
>> "Qty1" : 10,
>> "Qty2" : 14
>> }
>> {
>> "Date" : "4/12/2014",
>> "Person" : "Bob",
>> "Qty1" : 72,
>> "Qty2" : 33
>> }
>> {
>> "Date" : "5/1/2014",
>> "Person" : "Agnes",
>> "Qty1" : 24,
>> "Qty2" : 25
>> }
>> {
>> "Date" : "5/2/2014",
>> "Person" : "Sarah",
>> "Qty1" : 86,
>> "Qty2" : 4
>> }
>> {
>> "Date" : "5/6/2014",
>> "Person" : "Bob",
>> "Qty1" : 17,
>> "Qty2" : 4
>> }
>>
>> loaded with `mongoimport --db mine --collection test --drop --file
>> c:/tmp/work1a.json`
>>
>> Thank you for any pointers.
>>

Mime
View raw message