drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jinfeng Ni <jinfengn...@gmail.com>
Subject Re: Unforking Calcite...
Date Mon, 31 Aug 2015 16:58:10 GMT
Hackathon seems to be a brilliant idea. Definitely, it will help a lot if
we
could get Julian to join and help to review / work through these issues.



On Mon, Aug 31, 2015 at 9:33 AM, Jacques Nadeau <jacques@dremio.com> wrote:

> I'd like to propose that we do a Hackathon to try to close up the gap.
> Hopefully, we could get Julian to join and work through these issues with
> him. I'm sure that there is a way that we can either merge our patches
> (with different options) or change them to use new extension points. It is
> bad for everyone to maintain a fork so I think everyone will be equally
> invested in getting things merged.
>
> Thoughts?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Sun, Aug 30, 2015 at 9:49 PM, Jinfeng Ni <jinfengni99@gmail.com> wrote:
>
> > You are exactly right. We have 26 patches. Here they are
> > (--pretty=format:"%an %h %s")
> >
> > Jinfeng Ni 857ee95 VolcanoPlanner.clear() should clear ruleNames, in
> order
> > to avoid rule naming conflict error.
> > Jinfeng Ni ba2166f Make CalciteSchema extendable for different
> > implemetations by refactoring CalciteSchema and making it an interface
> > Hsuan-Yi Chu 046302d [CALCITE-827] Add a LogicalProject on top of
> > LogicalWindow to permute the columns in the same order as specified in
> the
> > SELECT list
> > Jinfeng Ni 2e076e1 Drill-specific change: Add back AbstractConverter in
> > RelSet.java and make sure AbstractConverter is not created when subset
> has
> > "NONE" conersion.
> > Mehant Baid 7ae335c Remove redundant invocation to ensureType in
> > RexBuilder.makeOver(). Don't invoke ensureType() for rewriting aggregate
> > expressions part of window functions.
> > Jinfeng Ni 3bcce80 CALCITE-628 related but not fix theproblem: Ensure
> > target traits are simple when use Frameworks or RelOptRule.convert()
> > method.
> > vkorukanti e1876f0 Add new method to ViewExpander interface to allow
> > passing SchemaRoot.
> > Jinfeng Ni 8c8acee Match Logical Rel in ReduceExpressionRule.
> > Jason Altekruse a29f007 Enable inheriting from previously singleton
> filter
> > and calc constant reduction rules.
> > Jason Altekruse 15d645f Make the consent expression Executor configurable
> > in FrameworkConfig.
> > Jinfeng Ni f176aa5 Disable the nullability cast checking in Filter.
> > Jinfeng Ni d0e2690 Add a new field name Suggester, so that duplicate
> names
> > will be renamed using the same policy as 0.9.
> > Jinfeng Ni 84bae71 Return validated row type and validated SqlNode when
> > call Planner.validate().
> > Aman Sinha 034e6e9 Workaround for CALCITE-528: Use case-insensitive
> > comparison when creating output row type of a JoinRel.
> > Aman Sinha 1f2fb1c Don't use 'SumEmptyIsZero' (SUM0) window aggregate
> until
> > CALCITE-777 is fixed.
> > Jinfeng Ni fcacd35 Make public the constructor of
> > SqlSumEmptyIsZeroAggFunction,
> > Aman Sinha 13d6449 Make certain push project constructors protected such
> > that derived classes can access them.
> > Aman Sinha aebbb34 DRILL-1455: Add return type-inference strategy for
> > arithmetic operators when one of the arguments is ANY type.
> > Mehant Baid 32accb4 Some changes related to Sql ANY type.
> > Jinfeng Ni e182e2c [StarColumn] Use case insensitive in function name
> > comparison, in order to fix Sql Validator Exception when group by a
> > function expression.
> > Jinfeng Ni b75b374 [StarColumn] Make sure expression in the select list
> is
> > expanded at most once.
> > Jinfeng Ni bca12a6 [StarColumn] Keep the original field names from left,
> > when rewrite a IN/EXISTS/NOT IN /NOT EXISTS subquery.  This is required
> to
> > support * column in schema-less system.
> > Jinfeng Ni eba9342 [StarColumn] Fix issues related to SqlIdentifier for
> > star column.
> > Hsuan-Yi Chu 70fcbca [StarColumn] When group-by a column, projecting on a
> > star which cannot be expanded at planning time, use ITEM operator to wrap
> > this column
> > Jinfeng Ni b3dc5b5 [StarColumn] Reverse one change in CALCITE-356, which
> > regresses AggChecker logic, after * query in schema-less table is added.
> > Jinfeng Ni 05fea03 [StarColumn] Support select * from schema-less table
> in
> > execution engine like Drill. Include support for view, subquery, CTE.
> >
> > I have pushed at least 2 patches to Calcite in the timeframe of
> > 1.4.0-release. While working on the rebasing work, I also try to wrap
> some
> > patches (including the "famous" calcite Schema change!) and make they
> ready
> > for pushing back to Calcite.
> >
> > In general, I agree that we should set a target to remove the forked
> > version. For the time frame of
> > 1.2 release, I will try to push at least 2 patches to Calcite master. I
> > also would like to have others
> > listed above spend sometime to push their corresponding patches to
> > Calcite.  For example, I talked
> > to Jason before, and he kindly agreed to push his two patches to Calcite
> (
> > they are pretty ready to
> > merge to calcite master). There is also [CALCITE-827], which has been
> > already in Calcite next
> > release, but not in 1.4.0-release.
> >
> > However, I'm not sure if we could reduce the # of patches to 13 in 1.2,
> and
> > especially reduce to 2
> > by 1.3 release.
> >   - Many patches are to solve Drill's specific issues, for instance, Star
> > column in schema-less
> >     system, AbstractConverter in VolcanoPlanner). That's why we even do
> not
> > create Calcite
> >     JIRA for them yet.  In other words, Calcite would work perfectly
> > without those patches, while
> >     Drill would fail lots of queries.
> >   - For some patches, we have tried to push to Calcite, but Calcite
> > community does not accept
> >    the patches. What should we do in such scenarios?
> >
> > I would welcome any help, if someone else would like to jump in and offer
> > his/her help.
> >
> > Regards,
> >
> > Jinfeng
> >
> >
> >
> >
> >
> > On Sun, Aug 30, 2015 at 4:59 PM, Jacques Nadeau <jacques@apache.org>
> > wrote:
> >
> > > Its great that we have rebased on top of Calcite 1.4.  However, it
> looks
> > > like we're still on a forked version of Calcite.  When I look here:
> > > https://github.com/mapr/incubator-calcite/commits/DrillCalcite1.4.0, I
> > > count ~26 patches that still need to be merged (many of them without
> > even a
> > > Calcite bug id).
> > >
> > > I think we need to have a target number of non-merged patches or we'll
> > > never get off the fork.  What do people think about bounding the
> release
> > on
> > > merging at least half of the items listed on that branch before we
> > release
> > > Drill 1.2 (~13) and we get down to no more than 2 for 1.3?
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message