calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesus Camacho Rodriguez <jcama...@apache.org>
Subject Re: Towards Calcite release 1.12
Date Tue, 14 Mar 2017 14:18:33 GMT
Julian,

I have deferred those three issues to next release since they are not blockers and I could
not work on them.

Once Nishant has addressed your comments in CALCITE-1683, I will merge his PR.

Thanks,
Jesús




On 3/14/17, 1:13 PM, "Remus Rusanu" <rrusanu@hortonworks.com> wrote:

>Hello Julian,
>
>I added an extra check and prevent constant reduction in the HiveExecutor for TIMESTAMPs
that have precision higher than 3. Not based on type, since Hive has always TIMESTAMP(9),
but on actual values that contain non-zero sub-millisecond nanos. This almost works, the problem
is that the expression still gets reduced at a later stage:
>
>	at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:1239)
>	at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:1236)
>	at org.apache.calcite.rex.RexExecutable.reduce(RexExecutable.java:86)
>	at org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:128)
>	at org.apache.calcite.rex.RexUtil.simplifyCast(RexUtil.java:2450)
>	at org.apache.calcite.rex.RexUtil.simplify(RexUtil.java:1633)
>	at org.apache.calcite.rex.RexUtil.simplify(RexUtil.java:1587)
>	at org.apache.calcite.rex.RexUtil.simplifyList(RexUtil.java:1747)
>	at org.apache.calcite.rex.RexUtil.simplifyComparison(RexUtil.java:1658)
>	at org.apache.calcite.rex.RexUtil.simplify(RexUtil.java:1648)
>	at org.apache.calcite.rex.RexUtil$ExprSimplifier.visitCall(RexUtil.java:3051)
>	at org.apache.calcite.rex.RexUtil$ExprSimplifier.visitCall(RexUtil.java:3016)
>	at org.apache.calcite.rex.RexCall.accept(RexCall.java:104)
>	at org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:279)
>
>Here the issue is that the custom HiveExecutor gets lost on the stack and the default
EXECUTOR is used instead, resulting in incorrect behavior. I created CALCITE-1695 Not all
RexUtil.simplifyXxx code paths carry the provided executor. This new CALCITE-1695 issue is
fairly trivial to fix. I know Julian talked about making the executor a property of the RexBuilder
and that would be better long term, but right now I tried just extending the signature of
simplifyComparison/SimplifyList to accept and pass on a RexExecutor and this seems to solve
the issues I was seeing in Hive. 
> 
>Thanks,
>~Remus
>
>
>On 3/13/17, 4:32 PM, "Julian Hyde" <jhyde@apache.org> wrote:
>
>    1690 is a big one. For now, can you just disable constant reduction for timestamp
literals that have high precision?
>    
>    > On Mar 13, 2017, at 4:30 PM, Remus Rusanu <rrusanu@hortonworks.com> wrote:
>    > 
>    > CALCITE-1681 is not needed anymore
>    > The only pending issue for Hive is CALCITE-1690: Calcite timestamp literals cannot
express precision above millisecond.
>    > 
>    > Thanks,
>    > ~Remus
>    > 
>    > ________________________________________
>    > From: Julian Hyde <jhyde@apache.org>
>    > Sent: Tuesday, March 14, 2017 1:23 AM
>    > To: dev@calcite.apache.org
>    > Subject: Re: Towards Calcite release 1.12
>    > 
>    > OK, what’s left before RC0?
>    > 
>    > I am working on committing 1561, 1684, 1691, 1688. Also, I will review 1566 shortly.
I may also get to 1689.
>    > 
>    > Remus, Are all of your Hive issues resolved? In particular do you need 1681?
>    > 
>    > Jesus, Where are you with 1421, 1456, 1413? Can you also commit Nishant’s 1683
since you +1’d it?
>    > 
>    > Julian
>    > 
>    > 
>    >> On Mar 8, 2017, at 12:25 PM, Julian Hyde <jhyde@apache.org> wrote:
>    >> 
>    >> We have that dependency already, albeit transitively: CALCITE-1681 blocks
HIVE-15708, which is depended upon by CALCITE-1657 “Release Calcite 1.12.0”.
>    >> 
>    >>> On Mar 8, 2017, at 12:12 PM, Remus Rusanu <rrusanu@hortonworks.com>
wrote:
>    >>> 
>    >>> Looks like CALCITE-1681 ‘Provide a way to copy RelNode trees between
clusters’ is also needed. I hope I’ll have it ready this week.
>    >>> 
>    >>> On 3/8/17, 10:49 AM, "Julian Hyde" <jhyde@apache.org> wrote:
>    >>> 
>    >>>  Same comments as for CALCITE-1615. The list contains commits up to 2/24.
I will fix before the release.
>    >>> 
>    >>>> On Mar 8, 2017, at 10:43 AM, Eli Levine <elilevine@gmail.com>
wrote:
>    >>>> 
>    >>>> https://issues.apache.org/jira/browse/CALCITE-1598 (Pig adapter)
does not
>    >>>> seem to be on the list.
>    >>>> 
>    >>>> Thanks,
>    >>>> 
>    >>>> Eli
>    >>>> 
>    >>>> 
>    >>>> On Wed, Mar 8, 2017 at 10:40 AM, Julian Hyde <jhyde@apache.org>
wrote:
>    >>>> 
>    >>>>> Commit 49888a6c581f6243a5b3f1a48ea2c9edd6aac9d5, which fixed
>    >>>>> CALCITE-1615, had not made it to https://github.com/julianhyde/
>    >>>>> calcite/tree/branch-1.12 <https://github.com/julianhyde/calcite/tree/
>    >>>>> branch-1.12> at the time I wrote that draft of the release
notes. There
>    >>>>> are about a dozen other cases fixed since then. I’ll be sure
to include it.
>    >>>>> 
>    >>>>> Julian
>    >>>>> 
>    >>>>> 
>    >>>>>> On Mar 8, 2017, at 10:29 AM, Haohui Mai <ricetons@gmail.com>
wrote:
>    >>>>>> 
>    >>>>>> Hi Julian,
>    >>>>>> 
>    >>>>>> I think CALCITE-1615 is not on the list. Can you please double
check?
>    >>>>>> 
>    >>>>>> Haohui
>    >>>>>> On Wed, Mar 8, 2017 at 9:49 AM Julian Hyde <jhyde@apache.org>
wrote:
>    >>>>>> 
>    >>>>>>> First draft of release notes:
>    >>>>>>> https://github.com/julianhyde/calcite/blob/branch-1.12/site/
>    >>>>> _docs/history.md
>    >>>>>>> <
>    >>>>>>> https://github.com/julianhyde/calcite/blob/branch-1.12/site/
>    >>>>> _docs/history.md>
>    >>>>>>> (contains commits up to 12 days ago).
>    >>>>>>> 
>    >>>>>>> 
>    >>>>>>>> On Mar 8, 2017, at 9:16 AM, Julian Hyde <jhyde@apache.org>
wrote:
>    >>>>>>>> 
>    >>>>>>>> Thanks. That last change should be fairly low-risk,
so we should be
>    >>>>> able
>    >>>>>>> to move to an RC and vote soon afterwards.
>    >>>>>>>> 
>    >>>>>>>> Committers, Let’s try to stabilize. I propose we
get any “risky”
>    >>>>> changes
>    >>>>>>> in by the end of the weekend.
>    >>>>>>>> 
>    >>>>>>>> By the way, I volunteer to be release manager.
>    >>>>>>>> 
>    >>>>>>>> Julian
>    >>>>>>>> 
>    >>>>>>>>> On Mar 8, 2017, at 2:53 AM, Remus Rusanu <rrusanu@hortonworks.com>
>    >>>>>>> wrote:
>    >>>>>>>>> 
>    >>>>>>>>> We have only one issue left to resolve, the materialized
views RelNode
>    >>>>>>> cloning into a new planner. I expect to have a resolution
by end of this
>    >>>>>>> week, so the all-done ETA from me would be mid-next week,
let say March
>    >>>>> 15.
>    >>>>>>>>> 
>    >>>>>>>>> Thanks,
>    >>>>>>>>> ~Remus
>    >>>>>>>>> 
>    >>>>>>>>> On 3/7/17, 5:08 PM, "Julian Hyde" <jhyde@apache.org>
wrote:
>    >>>>>>>>> 
>    >>>>>>>>> Remus,
>    >>>>>>>>> 
>    >>>>>>>>> Do you have an ETA for HIVE-15708? Just a guess
is fine, but it helps
>    >>>>>>> the rest of us know when this release is likely to drop.
>    >>>>>>>>> 
>    >>>>>>>>> Julian
>    >>>>>>>>> 
>    >>>>>>>>>> On Feb 28, 2017, at 10:57 AM, Julian Hyde
<jhyde@apache.org> wrote:
>    >>>>>>>>>> 
>    >>>>>>>>>> On Tue, Feb 28, 2017 at 1:40 AM, Remus Rusanu
<
>    >>>>> rrusanu@hortonworks.com>
>    >>>>>>> wrote:
>    >>>>>>>>>>> Put on hold, for how long?
>    >>>>>>>>>> 
>    >>>>>>>>>> Until someone chimed in with their time constraints,
as you just did.
>    >>>>>>> :)
>    >>>>>>>>>> 
>    >>>>>>>>>> We make decisions on-list, but for that we
need people to provide
>    >>>>>>>>>> information on-list.
>    >>>>>>>>>> 
>    >>>>>>>>>>> [We] did not yet have a 100% successful
run for HIVE-15708,
>    >>>>>>>>>>> but we're close and once we have the
changes ready, we
>    >>>>>>>>>>> cannot do the move unless Calcite does
the release.
>    >>>>>>>>>> 
>    >>>>>>>>>> What's your best guess for when HIVE-15708
will be fixed? We will
>    >>>>>>>>>> start the RC vote after that.
>    >>>>>>>>>> 
>    >>>>>>>>>> Julian
>    >>>>>>>>> 
>    >>>>>>>>> 
>    >>>>>>>>> 
>    >>>>>>>>> 
>    >>>>>>>> 
>    >>>>>>> 
>    >>>>>>> 
>    >>>>> 
>    >>>>> 
>    >>> 
>    >>> 
>    >>> 
>    >>> 
>    >> 
>    > 
>    > 
>    
>    
>    
>


Mime
View raw message