drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-2852) CASTing the column 'dir0' in view causes partition pruning to fail
Date Tue, 04 Aug 2015 02:04:04 GMT

    [ https://issues.apache.org/jira/browse/DRILL-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652957#comment-14652957
] 

Aman Sinha commented on DRILL-2852:
-----------------------------------

Processing arbitrary expressions that involve the directory columns will require some thought
and may be non-trivial.  Consider dir0 - 1 + 5 * 2 = 1994 - 2.   Ideally, we would want to
transform this to  dir0 = <some constant>  because we can not  be sure that 2 or more
directory columns are not occurring in the same expression.  In order to transform to  dirN
= <some constant> we would have to call the constant reduction rule. 
Another consideration is that dir0  type may not be numeric, so doing arithmetic operation
won't be valid and the interpreter evaluator will fail but the PruneScan rule would have done
some wasted work. 

Ignoring expressions for now, I think we could potentially support single-input functions
that involve the directory column; however, it would be better to get some use cases around
this.  

> CASTing the column 'dir0' in view causes partition pruning to fail 
> -------------------------------------------------------------------
>
>                 Key: DRILL-2852
>                 URL: https://issues.apache.org/jira/browse/DRILL-2852
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 0.8.0
>            Reporter: Aman Sinha
>            Assignee: Jinfeng Ni
>             Fix For: 1.2.0
>
>
> If the partition column 'dir0' is CAST inside a view and the query has a filter on that
column, then partition pruning interpreter evaluator encounters an AssertionError. 
> Table data in the example below is from test/resources/multilevel/parquet.  
> {code}
> create view dfs.tmp.myview2 as select cast(dir0 as varchar(100)) as myyear, dir1 as myquarter,
o_totalprice from `multilevel/parquet` ;
> select * from dfs.tmp.myview2 where myyear = '1995' and myquarter = 'Q2' and o_totalprice
< 40000.0;
> {code}
> Failure stack trace: 
> {code}
> Caused by: java.lang.AssertionError: Internal error: Error while applying rule PruneScanRule:Filter_On_Project,
> ...
> <skip>
> Caused by: java.lang.AssertionError: null
>         at org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.isBitOn(InterpreterEvaluator.java:490)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
>         at org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanAnd(InterpreterEvaluator.java:434)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
>         at org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanOperator(InterpreterEvaluator.java:332)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
>         at org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanOperator(InterpreterEvaluator.java:147)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
>         at org.apache.drill.common.expression.BooleanOperator.accept(BooleanOperator.java:36)
~[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
>         at org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluate(InterpreterEvaluator.java:80)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
>         at org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:224)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message