calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: Towards Calcite release 1.12
Date Wed, 15 Mar 2017 18:10:17 GMT
All the blockers have been resolved (see the tracking issue,
https://issues.apache.org/jira/browse/CALCITE-1657). I just pushed a
maven snapshot based on commit
https://github.com/apache/calcite/commit/ca48431ee7872f294de7a10a6da0892185e8e4a7.
We're almost good to go for a first release candidate.

Are there any more blockers? (I'm going to review
https://issues.apache.org/jira/browse/CALCITE-1566 but it's only a
documentation change, so can't break anything.)

If your project depends upon Calcite, please try running it against
the snapshot. Let us know if there are issues.

Julian


On Tue, Mar 14, 2017 at 7:18 AM, Jesus Camacho Rodriguez
<jcamacho@apache.org> wrote:
> Julian,
>
> I have deferred those three issues to next release since they are not blockers and I
could not work on them.
>
> Once Nishant has addressed your comments in CALCITE-1683, I will merge his PR.
>
> Thanks,
> Jesús
>
>
>
>
> On 3/14/17, 1:13 PM, "Remus Rusanu" <rrusanu@hortonworks.com> wrote:
>
>>Hello Julian,
>>
>>I added an extra check and prevent constant reduction in the HiveExecutor for TIMESTAMPs
that have precision higher than 3. Not based on type, since Hive has always TIMESTAMP(9),
but on actual values that contain non-zero sub-millisecond nanos. This almost works, the problem
is that the expression still gets reduced at a later stage:
>>
>>       at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:1239)
>>       at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:1236)
>>       at org.apache.calcite.rex.RexExecutable.reduce(RexExecutable.java:86)
>>       at org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:128)
>>       at org.apache.calcite.rex.RexUtil.simplifyCast(RexUtil.java:2450)
>>       at org.apache.calcite.rex.RexUtil.simplify(RexUtil.java:1633)
>>       at org.apache.calcite.rex.RexUtil.simplify(RexUtil.java:1587)
>>       at org.apache.calcite.rex.RexUtil.simplifyList(RexUtil.java:1747)
>>       at org.apache.calcite.rex.RexUtil.simplifyComparison(RexUtil.java:1658)
>>       at org.apache.calcite.rex.RexUtil.simplify(RexUtil.java:1648)
>>       at org.apache.calcite.rex.RexUtil$ExprSimplifier.visitCall(RexUtil.java:3051)
>>       at org.apache.calcite.rex.RexUtil$ExprSimplifier.visitCall(RexUtil.java:3016)
>>       at org.apache.calcite.rex.RexCall.accept(RexCall.java:104)
>>       at org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:279)
>>
>>Here the issue is that the custom HiveExecutor gets lost on the stack and the default
EXECUTOR is used instead, resulting in incorrect behavior. I created CALCITE-1695 Not all
RexUtil.simplifyXxx code paths carry the provided executor. This new CALCITE-1695 issue is
fairly trivial to fix. I know Julian talked about making the executor a property of the RexBuilder
and that would be better long term, but right now I tried just extending the signature of
simplifyComparison/SimplifyList to accept and pass on a RexExecutor and this seems to solve
the issues I was seeing in Hive.
>>
>>Thanks,
>>~Remus
>>
>>
>>On 3/13/17, 4:32 PM, "Julian Hyde" <jhyde@apache.org> wrote:
>>
>>    1690 is a big one. For now, can you just disable constant reduction for timestamp
literals that have high precision?
>>
>>    > On Mar 13, 2017, at 4:30 PM, Remus Rusanu <rrusanu@hortonworks.com>
wrote:
>>    >
>>    > CALCITE-1681 is not needed anymore
>>    > The only pending issue for Hive is CALCITE-1690: Calcite timestamp literals
cannot express precision above millisecond.
>>    >
>>    > Thanks,
>>    > ~Remus
>>    >
>>    > ________________________________________
>>    > From: Julian Hyde <jhyde@apache.org>
>>    > Sent: Tuesday, March 14, 2017 1:23 AM
>>    > To: dev@calcite.apache.org
>>    > Subject: Re: Towards Calcite release 1.12
>>    >
>>    > OK, what’s left before RC0?
>>    >
>>    > I am working on committing 1561, 1684, 1691, 1688. Also, I will review 1566
shortly. I may also get to 1689.
>>    >
>>    > Remus, Are all of your Hive issues resolved? In particular do you need 1681?
>>    >
>>    > Jesus, Where are you with 1421, 1456, 1413? Can you also commit Nishant’s
1683 since you +1’d it?
>>    >
>>    > Julian
>>    >
>>    >
>>    >> On Mar 8, 2017, at 12:25 PM, Julian Hyde <jhyde@apache.org> wrote:
>>    >>
>>    >> We have that dependency already, albeit transitively: CALCITE-1681 blocks
HIVE-15708, which is depended upon by CALCITE-1657 “Release Calcite 1.12.0”.
>>    >>
>>    >>> On Mar 8, 2017, at 12:12 PM, Remus Rusanu <rrusanu@hortonworks.com>
wrote:
>>    >>>
>>    >>> Looks like CALCITE-1681 ‘Provide a way to copy RelNode trees between
clusters’ is also needed. I hope I’ll have it ready this week.
>>    >>>
>>    >>> On 3/8/17, 10:49 AM, "Julian Hyde" <jhyde@apache.org> wrote:
>>    >>>
>>    >>>  Same comments as for CALCITE-1615. The list contains commits up
to 2/24. I will fix before the release.
>>    >>>
>>    >>>> On Mar 8, 2017, at 10:43 AM, Eli Levine <elilevine@gmail.com>
wrote:
>>    >>>>
>>    >>>> https://issues.apache.org/jira/browse/CALCITE-1598 (Pig adapter)
does not
>>    >>>> seem to be on the list.
>>    >>>>
>>    >>>> Thanks,
>>    >>>>
>>    >>>> Eli
>>    >>>>
>>    >>>>
>>    >>>> On Wed, Mar 8, 2017 at 10:40 AM, Julian Hyde <jhyde@apache.org>
wrote:
>>    >>>>
>>    >>>>> Commit 49888a6c581f6243a5b3f1a48ea2c9edd6aac9d5, which fixed
>>    >>>>> CALCITE-1615, had not made it to https://github.com/julianhyde/
>>    >>>>> calcite/tree/branch-1.12 <https://github.com/julianhyde/calcite/tree/
>>    >>>>> branch-1.12> at the time I wrote that draft of the release
notes. There
>>    >>>>> are about a dozen other cases fixed since then. I’ll be
sure to include it.
>>    >>>>>
>>    >>>>> Julian
>>    >>>>>
>>    >>>>>
>>    >>>>>> On Mar 8, 2017, at 10:29 AM, Haohui Mai <ricetons@gmail.com>
wrote:
>>    >>>>>>
>>    >>>>>> Hi Julian,
>>    >>>>>>
>>    >>>>>> I think CALCITE-1615 is not on the list. Can you please
double check?
>>    >>>>>>
>>    >>>>>> Haohui
>>    >>>>>> On Wed, Mar 8, 2017 at 9:49 AM Julian Hyde <jhyde@apache.org>
wrote:
>>    >>>>>>
>>    >>>>>>> First draft of release notes:
>>    >>>>>>> https://github.com/julianhyde/calcite/blob/branch-1.12/site/
>>    >>>>> _docs/history.md
>>    >>>>>>> <
>>    >>>>>>> https://github.com/julianhyde/calcite/blob/branch-1.12/site/
>>    >>>>> _docs/history.md>
>>    >>>>>>> (contains commits up to 12 days ago).
>>    >>>>>>>
>>    >>>>>>>
>>    >>>>>>>> On Mar 8, 2017, at 9:16 AM, Julian Hyde <jhyde@apache.org>
wrote:
>>    >>>>>>>>
>>    >>>>>>>> Thanks. That last change should be fairly low-risk,
so we should be
>>    >>>>> able
>>    >>>>>>> to move to an RC and vote soon afterwards.
>>    >>>>>>>>
>>    >>>>>>>> Committers, Let’s try to stabilize. I propose
we get any “risky”
>>    >>>>> changes
>>    >>>>>>> in by the end of the weekend.
>>    >>>>>>>>
>>    >>>>>>>> By the way, I volunteer to be release manager.
>>    >>>>>>>>
>>    >>>>>>>> Julian
>>    >>>>>>>>
>>    >>>>>>>>> On Mar 8, 2017, at 2:53 AM, Remus Rusanu
<rrusanu@hortonworks.com>
>>    >>>>>>> wrote:
>>    >>>>>>>>>
>>    >>>>>>>>> We have only one issue left to resolve, the
materialized views RelNode
>>    >>>>>>> cloning into a new planner. I expect to have a resolution
by end of this
>>    >>>>>>> week, so the all-done ETA from me would be mid-next
week, let say March
>>    >>>>> 15.
>>    >>>>>>>>>
>>    >>>>>>>>> Thanks,
>>    >>>>>>>>> ~Remus
>>    >>>>>>>>>
>>    >>>>>>>>> On 3/7/17, 5:08 PM, "Julian Hyde" <jhyde@apache.org>
wrote:
>>    >>>>>>>>>
>>    >>>>>>>>> Remus,
>>    >>>>>>>>>
>>    >>>>>>>>> Do you have an ETA for HIVE-15708? Just a
guess is fine, but it helps
>>    >>>>>>> the rest of us know when this release is likely to
drop.
>>    >>>>>>>>>
>>    >>>>>>>>> Julian
>>    >>>>>>>>>
>>    >>>>>>>>>> On Feb 28, 2017, at 10:57 AM, Julian
Hyde <jhyde@apache.org> wrote:
>>    >>>>>>>>>>
>>    >>>>>>>>>> On Tue, Feb 28, 2017 at 1:40 AM, Remus
Rusanu <
>>    >>>>> rrusanu@hortonworks.com>
>>    >>>>>>> wrote:
>>    >>>>>>>>>>> Put on hold, for how long?
>>    >>>>>>>>>>
>>    >>>>>>>>>> Until someone chimed in with their time
constraints, as you just did.
>>    >>>>>>> :)
>>    >>>>>>>>>>
>>    >>>>>>>>>> We make decisions on-list, but for that
we need people to provide
>>    >>>>>>>>>> information on-list.
>>    >>>>>>>>>>
>>    >>>>>>>>>>> [We] did not yet have a 100% successful
run for HIVE-15708,
>>    >>>>>>>>>>> but we're close and once we have
the changes ready, we
>>    >>>>>>>>>>> cannot do the move unless Calcite
does the release.
>>    >>>>>>>>>>
>>    >>>>>>>>>> What's your best guess for when HIVE-15708
will be fixed? We will
>>    >>>>>>>>>> start the RC vote after that.
>>    >>>>>>>>>>
>>    >>>>>>>>>> Julian
>>    >>>>>>>>>
>>    >>>>>>>>>
>>    >>>>>>>>>
>>    >>>>>>>>>
>>    >>>>>>>>
>>    >>>>>>>
>>    >>>>>>>
>>    >>>>>
>>    >>>>>
>>    >>>
>>    >>>
>>    >>>
>>    >>>
>>    >>
>>    >
>>    >
>>
>>
>>
>>
>

Mime
View raw message