drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jinfeng Ni (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5359) ClassCastException when push down filter on the output of flatten into parquet scan
Date Thu, 16 Mar 2017 21:41:41 GMT
Jinfeng Ni created DRILL-5359:
---------------------------------

             Summary: ClassCastException when push down filter on the output of flatten into
parquet scan
                 Key: DRILL-5359
                 URL: https://issues.apache.org/jira/browse/DRILL-5359
             Project: Apache Drill
          Issue Type: Bug
          Components: Query Planning & Optimization
    Affects Versions: 1.9.0
            Reporter: Jinfeng Ni
            Assignee: Jinfeng Ni
             Fix For: 1.11.0


The following simplified query would hit ClassCastException.

{code}
select n_regionkey
from (select n_regionkey, 
            flatten(nation.cities) as cities 
      from cp.`tpch/nation.parquet` nation) as flattenedCities 
where flattenedCities.cities.`zip` = '12345';
{code}

Here is the stacktrace for the Exception : 
{code}
caused by: java.lang.ClassCastException: org.apache.drill.common.expression.FunctionCall cannot
be cast to org.apache.drill.common.expression.SchemaPath
	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:170)
~[classes/:na]
	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:80)
~[classes/:na]
	at org.apache.calcite.rex.RexCall.accept(RexCall.java:107) ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.doFunction(DrillOptiq.java:205)
~[classes/:na]
	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:105)
~[classes/:na]
	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:80)
~[classes/:na]
	at org.apache.calcite.rex.RexCall.accept(RexCall.java:107) ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
	at org.apache.drill.exec.planner.logical.DrillOptiq.toDrill(DrillOptiq.java:77) ~[classes/:na]
	at org.apache.drill.exec.store.parquet.ParquetPushDownFilter.doOnMatch(ParquetPushDownFilter.java:141)
~[classes/:na]
	at org.apache.drill.exec.store.parquet.ParquetPushDownFilter$1.onMatch(ParquetPushDownFilter.java:68)
~[classes/:na]
{code}

The cause of this problem: Parquet filter pushdown rule tries to push a filter expression
containing item/flatten operators into parquet scan. However, the method DrillOptiq.toDrill()
does not allow such expression (since "flatten" is not a scalar function).  

The solution is to disable pushing such filter expression.  Even the rule allows, the underneath
parquet metadata would not have the corresponding statistics; there is no point to considering
push such filter expression. 




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message