drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Volodymyr Vysotskyi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-6193) Latest Calcite optimized out join condition and cause "This query cannot be planned possibly due to either a cartesian join or an inequality join"
Date Wed, 28 Feb 2018 09:58:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16380071#comment-16380071
] 

Volodymyr Vysotskyi commented on DRILL-6193:
--------------------------------------------

This simplification of filter condition happens during the creation of new filter when used
{{RelBuilder.filter()}} method. With Calcite upgrade, we started using {{DrillRelBuilder}}
instead of Calcites {{RelBuilder}}.

To fix this issue we can override {{RelBuilder.filter()}} method in {{DrillRelBuilder}} and
after simplification of filter condition split AND predicates, check and add back if needed
every predicate if it can be used in the join condition.

Here is an example how to determine if equals predicate may be used in the join: [https://github.com/apache/calcite/commit/b60b67eb8f62463ccbc230358969ef2450cdbe05?diff=unified#diff-a6a937c185ffdee97b49b98530c5112dR713].

> Latest Calcite optimized out join condition and cause "This query cannot be planned possibly
due to either a cartesian join or an inequality join"
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-6193
>                 URL: https://issues.apache.org/jira/browse/DRILL-6193
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning &amp; Optimization
>    Affects Versions: 1.13.0
>            Reporter: Chunhui Shi
>            Assignee: Hanumath Rao Maduri
>            Priority: Blocker
>             Fix For: 1.13.0
>
>
> I got the same error on apache master's MapR profile on the tip(before Hive upgrade)
and on changeset 9e944c97ee6f6c0d1705f09d531af35deed2e310, the last commit of Calcite upgrade
with the failed query reported in functional test but now it is on parquet file:
>  
> {quote}SELECT L.L_QUANTITY, L.L_DISCOUNT, L.L_EXTENDEDPRICE, L.L_TAX
>  
> FROM cp.`tpch/lineitem.parquet` L, cp.`tpch/orders.parquet` O
> WHERE cast(L.L_ORDERKEY as int) = cast(O.O_ORDERKEY as int) AND cast(L.L_LINENUMBER as
int) = 7 AND cast(L.L_ORDERKEY as int) = 10208 AND cast(O.O_ORDERKEY as int) = 10208;
>  {quote}
> However, built Drill on commit ef0fafea214e866556fa39c902685d48a56001e1, the commit
right before Calcite upgrade commits, the same query worked.
> This was caused by latest Calcite simplified the predicates and during this process,
"cast(L.L_ORDERKEY as int) = cast(O.O_ORDERKEY as int) " was considered redundant and was
removed, so the logical plan of this query is getting an always true condition for Join:
> {quote}DrillJoinRel(condition=[true], joinType=[inner])
> {quote}
> While in previous version we have 
> {quote}DrillJoinRel(condition=[=($5, $0)], joinType=[inner])
> {quote}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message