drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: Unforking Calcite...
Date Mon, 31 Aug 2015 16:33:59 GMT
I'd like to propose that we do a Hackathon to try to close up the gap.
Hopefully, we could get Julian to join and work through these issues with
him. I'm sure that there is a way that we can either merge our patches
(with different options) or change them to use new extension points. It is
bad for everyone to maintain a fork so I think everyone will be equally
invested in getting things merged.


Jacques Nadeau
CTO and Co-Founder, Dremio

On Sun, Aug 30, 2015 at 9:49 PM, Jinfeng Ni <jinfengni99@gmail.com> wrote:

> You are exactly right. We have 26 patches. Here they are
> (--pretty=format:"%an %h %s")
> Jinfeng Ni 857ee95 VolcanoPlanner.clear() should clear ruleNames, in order
> to avoid rule naming conflict error.
> Jinfeng Ni ba2166f Make CalciteSchema extendable for different
> implemetations by refactoring CalciteSchema and making it an interface
> Hsuan-Yi Chu 046302d [CALCITE-827] Add a LogicalProject on top of
> LogicalWindow to permute the columns in the same order as specified in the
> SELECT list
> Jinfeng Ni 2e076e1 Drill-specific change: Add back AbstractConverter in
> RelSet.java and make sure AbstractConverter is not created when subset has
> "NONE" conersion.
> Mehant Baid 7ae335c Remove redundant invocation to ensureType in
> RexBuilder.makeOver(). Don't invoke ensureType() for rewriting aggregate
> expressions part of window functions.
> Jinfeng Ni 3bcce80 CALCITE-628 related but not fix theproblem: Ensure
> target traits are simple when use Frameworks or RelOptRule.convert()
> method.
> vkorukanti e1876f0 Add new method to ViewExpander interface to allow
> passing SchemaRoot.
> Jinfeng Ni 8c8acee Match Logical Rel in ReduceExpressionRule.
> Jason Altekruse a29f007 Enable inheriting from previously singleton filter
> and calc constant reduction rules.
> Jason Altekruse 15d645f Make the consent expression Executor configurable
> in FrameworkConfig.
> Jinfeng Ni f176aa5 Disable the nullability cast checking in Filter.
> Jinfeng Ni d0e2690 Add a new field name Suggester, so that duplicate names
> will be renamed using the same policy as 0.9.
> Jinfeng Ni 84bae71 Return validated row type and validated SqlNode when
> call Planner.validate().
> Aman Sinha 034e6e9 Workaround for CALCITE-528: Use case-insensitive
> comparison when creating output row type of a JoinRel.
> Aman Sinha 1f2fb1c Don't use 'SumEmptyIsZero' (SUM0) window aggregate until
> CALCITE-777 is fixed.
> Jinfeng Ni fcacd35 Make public the constructor of
> SqlSumEmptyIsZeroAggFunction,
> Aman Sinha 13d6449 Make certain push project constructors protected such
> that derived classes can access them.
> Aman Sinha aebbb34 DRILL-1455: Add return type-inference strategy for
> arithmetic operators when one of the arguments is ANY type.
> Mehant Baid 32accb4 Some changes related to Sql ANY type.
> Jinfeng Ni e182e2c [StarColumn] Use case insensitive in function name
> comparison, in order to fix Sql Validator Exception when group by a
> function expression.
> Jinfeng Ni b75b374 [StarColumn] Make sure expression in the select list is
> expanded at most once.
> Jinfeng Ni bca12a6 [StarColumn] Keep the original field names from left,
> when rewrite a IN/EXISTS/NOT IN /NOT EXISTS subquery.  This is required to
> support * column in schema-less system.
> Jinfeng Ni eba9342 [StarColumn] Fix issues related to SqlIdentifier for
> star column.
> Hsuan-Yi Chu 70fcbca [StarColumn] When group-by a column, projecting on a
> star which cannot be expanded at planning time, use ITEM operator to wrap
> this column
> Jinfeng Ni b3dc5b5 [StarColumn] Reverse one change in CALCITE-356, which
> regresses AggChecker logic, after * query in schema-less table is added.
> Jinfeng Ni 05fea03 [StarColumn] Support select * from schema-less table in
> execution engine like Drill. Include support for view, subquery, CTE.
> I have pushed at least 2 patches to Calcite in the timeframe of
> 1.4.0-release. While working on the rebasing work, I also try to wrap some
> patches (including the "famous" calcite Schema change!) and make they ready
> for pushing back to Calcite.
> In general, I agree that we should set a target to remove the forked
> version. For the time frame of
> 1.2 release, I will try to push at least 2 patches to Calcite master. I
> also would like to have others
> listed above spend sometime to push their corresponding patches to
> Calcite.  For example, I talked
> to Jason before, and he kindly agreed to push his two patches to Calcite (
> they are pretty ready to
> merge to calcite master). There is also [CALCITE-827], which has been
> already in Calcite next
> release, but not in 1.4.0-release.
> However, I'm not sure if we could reduce the # of patches to 13 in 1.2, and
> especially reduce to 2
> by 1.3 release.
>   - Many patches are to solve Drill's specific issues, for instance, Star
> column in schema-less
>     system, AbstractConverter in VolcanoPlanner). That's why we even do not
> create Calcite
>     JIRA for them yet.  In other words, Calcite would work perfectly
> without those patches, while
>     Drill would fail lots of queries.
>   - For some patches, we have tried to push to Calcite, but Calcite
> community does not accept
>    the patches. What should we do in such scenarios?
> I would welcome any help, if someone else would like to jump in and offer
> his/her help.
> Regards,
> Jinfeng
> On Sun, Aug 30, 2015 at 4:59 PM, Jacques Nadeau <jacques@apache.org>
> wrote:
> > Its great that we have rebased on top of Calcite 1.4.  However, it looks
> > like we're still on a forked version of Calcite.  When I look here:
> > https://github.com/mapr/incubator-calcite/commits/DrillCalcite1.4.0, I
> > count ~26 patches that still need to be merged (many of them without
> even a
> > Calcite bug id).
> >
> > I think we need to have a target number of non-merged patches or we'll
> > never get off the fork.  What do people think about bounding the release
> on
> > merging at least half of the items listed on that branch before we
> release
> > Drill 1.2 (~13) and we get down to no more than 2 for 1.3?
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message