calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amogh Margoor <amo...@qubole.com>
Subject Re: Cluster mismatch between RelNodes of a query and a materialized view
Date Wed, 08 Mar 2017 17:24:15 GMT
We had created MaterializedCluster by extending RelOptCluster for solving
this issue :

https://github.com/qubole/quark/blob/master/optimizer/src/main/java/org/apache/calcite/plan/QuarkMaterializeCluster.java
New cluster was used in RelNodes of Materialised Views and it allowed
changing it's RelOptPlanner to that of RelNode being optimised. Caveat is
that it will not work if there are concurrent optimisations of RelNodes
happening in different threads using same copy of MaterializedViews.

Thanks,
Amogh


On Wed, Mar 8, 2017 at 10:35 PM, Julian Hyde <jhyde@apache.org> wrote:

> The argument should be a RelOptCluster, not a RelOptPlanner. The link from
> RelNode to planner is indirect currently (via cluster) and will be
> non-existent after CALCITE-1536.
>
> I question whether we need a new method. Putting an abstract method on
> RelNode is a huge burden because every RelNode sub-class needs to be fixed
> when people upgrade. Even a non-abstract method imposes a conceptual
> burden: more methods to understand.
>
> So, my approach would be to sub-class RelShuttle. It’s sufficient that it
> only works for LogicalXxx nodes.
>
> No need to copy RexNode expressions. They are immutable.
>
> Julian
>
>
> > On Mar 8, 2017, at 4:14 AM, Remus Rusanu <rrusanu@hortonworks.com>
> wrote:
> >
> > I created CALCITE-1681 https://issues.apache.org/
> jira/browse/CALCITE-1681 and I intent to work on it for finishing
> HIVE-15708.
> > My current thinking is to create a RelCopier based on RelShuttle and add
> a new abstract RelNode.copyTo(RelOptPlanner) that each concrete Rel type
> must override. The Rex part is already handled by the existing RexCopier.
> >
> > Thanks,
> > ~Remus
> >
> > On 3/6/17, 12:30 PM, "Julian Hyde" <jhyde@apache.org> wrote:
> >
> >    Every RelNode belongs to a RelOptCluster, and basically there is one
> RelOptCluster created each time a query is prepared. When working with
> materialized views, the view’s query is represented as a tree of RelNodes,
> that tree is used for optimizing more than one query. When planning a
> particular query, the nodes of that query will have a different
> RelOptCluster than the nodes of the materialized view(s) they are matched
> against.
> >
> >    How do we deal with this? Do we copy the nodes into the query’s
> cluster once we have found a match? If so, how? I couldn’t find a sub-class
> of RelVisitor or RelShuttle that copies trees to a different RelOptCluster.
> >
> >    By the way, https://issues.apache.org/jira/browse/CALCITE-1536 <
> https://issues.apache.org/jira/browse/CALCITE-1536> aims to improve the
> RelNode life-cycle but I don’t think it will solve this problem.
> >
> >    Julian
> >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message