calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Danny Chan <yuzhao....@gmail.com>
Subject Re: [DISCUSS] Support Sql Hint for Calcite
Date Fri, 18 Oct 2019 07:52:08 GMT
Thanks Julian, for current patch, I choose 1 because it can applied both to Hep and Volcano.
I make these latest changes:

• Add a new interface RelOptRuleCall.transformTo(RelNode, Map, BiFunction) to make the hints
copy strategy overridable
• Cache the hint strategies into RelOptCluster so that user can query the strategies during
rule planning


Another reason I didn’t choose 2 is that a RelNode’s parent node may also be derived from
a Rule matching, so I have to lookup recursively to find the real original node for the hints.

Best,
Danny Chan
在 2019年10月18日 +0800 AM4:55,Julian Hyde <jhyde@apache.org>,写道:
> I wonder whether it is possible to add some kind of “action handler” to the planner
engine, called, for example, when a rule has fired and is registering the RelNode created
by the rule. People can write their own action handlers to copy hints around. Since the action
handlers are the user’s code, they can iterate faster to find a hint-propagation strategy
that works in practice.
>
> Another idea is to use VolcanoPlanner.Provenance[1]. A RelNode can find its ancestor
RelNodes, and the rules that fired to create it. So it can grab hints from those ancestors.
It does not need to copy those hints onto itself.
>
> Julian
>
> [1] https://calcite.apache.org/apidocs/org/apache/calcite/plan/volcano/VolcanoPlanner.Provenance.html
<https://calcite.apache.org/apidocs/org/apache/calcite/plan/volcano/VolcanoPlanner.Provenance.html>
>
> > On Oct 16, 2019, at 8:38 PM, Haisheng Yuan <h.yuan@alibaba-inc.com> wrote:
> >
> > Julian,
> > Your concern is very valid, and that is also our main concern.
> > I was thinking whether we can put hint into the MEMO group, so that both logical
and physical expression in the same group can share the same hint, without copying the hint
explicitly. But for newly generated expression that doesn't belong to the original group,
we still need to copy hints. What's worse, in HepPlanner, there is no such concept, we may
still need to copy hints explicity in planner rules, if we want to keep the hint, which is
burdensome.
> >
> > - Haisheng
> >
> > ------------------------------------------------------------------
> > 发件人:Danny Chan<yuzhao.cyz@gmail.com>
> > 日 期:2019年10月16日 14:54:46
> > 收件人:<dev@calcite.apache.org>
> > 主 题:Re: [DISCUSS] Support Sql Hint for Calcite
> >
> > Thanks for the clarification.
> >
> > I understand you worried. Yes, the effort/memory would be wasted or meaningless
if hints are not used. This is just what a hint does, it is a “hint” and non-mandatory,
but we should give the chance to let user see them, it is the use that decide if to use the
hints and how to use them. For big queries I have no confidence to cover the corner cases.
So can we mark this feature as experimental and used for simple queries(no decorrelation)
first ?
> >
> > For “reversible”, during the implementation, I try to make the modifications
non-invasive with the current codes. That is why I made all the interfaces about the hint
into one class named RelWithHInt. Different with trait, I didn’t force users to pass in
the hints in the RelNode constructor. I think if is not a bigwork if we want to remove the
API.
> >
> > Best,
> > Danny Chan
> > 在 2019年10月16日 +0800 AM11:14,Julian Hyde <jhyde@apache.org>,写道:
> > > By “skeptical” I mean that I think we can come up with a mechanism to copy
hints when applying planner rules, but even when we have implemented that mechanism there
will be many cases where people want a hint and that hint is not copied to the RelNode where
it is needed, and many other cases where we spend the effort/memory of copying the hint to
a RelNode and the hint is not used.
> > >
> > > By “reversible” I mean if we come up with an API that does not work, how
do we change or remove that API without people complaining?
> > >
> > > Julian
> > >
> > >
> > > > On Oct 15, 2019, at 7:11 PM, Danny Chan <yuzhao.cyz@gmail.com> wrote:
> > > >
> > > > Thanks Julian
> > > >
> > > > > I am skeptical that RelWithHint will work for large queries.
> > > >
> > > > For “skeptical” do you mean how to transfer the hints during rule
planning ? I’m also not that confident yet.
> > > >
> > > > > How do we introduce it in a reversible way
> > > > Do you mean transform the RelWithHint back into the SqlHint ? I didn’t
implement it in current patch, but I think we have the ability to do that because we have
a inheritPath for each RelWithHint, we can collect all the hints together and merge them into
the SqlHints, then propagate these SqlHints to the SqlNodes.
> > > >
> > > > > What are the other options?
> > > > Do you mean the way to transfer hints during planning ? I have no other
options yet.
> > > >
> > > > Best,
> > > > Danny Chan
> > > > 在 2019年10月16日 +0800 AM8:03,dev@calcite.apache.org,写道:
> > > > >
> > > > > I am skeptical that RelWithHint will work for large queries.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message