drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hsuan Yi Chu <hyi...@maprtech.com>
Subject Re: Unforking Calcite...
Date Mon, 31 Aug 2015 22:44:02 GMT
The idea of Hackathon sounds pretty helpful. I would love to help too.



On Mon, Aug 31, 2015 at 3:41 PM, Ted Dunning <ted.dunning@gmail.com> wrote:

> We have a place to do it, in case that helps.
>
>
>
> On Mon, Aug 31, 2015 at 9:58 AM, Jinfeng Ni <jinfengni99@gmail.com> wrote:
>
> > Hackathon seems to be a brilliant idea. Definitely, it will help a lot if
> > we
> > could get Julian to join and help to review / work through these issues.
> >
> >
> >
> > On Mon, Aug 31, 2015 at 9:33 AM, Jacques Nadeau <jacques@dremio.com>
> > wrote:
> >
> > > I'd like to propose that we do a Hackathon to try to close up the gap.
> > > Hopefully, we could get Julian to join and work through these issues
> with
> > > him. I'm sure that there is a way that we can either merge our patches
> > > (with different options) or change them to use new extension points. It
> > is
> > > bad for everyone to maintain a fork so I think everyone will be equally
> > > invested in getting things merged.
> > >
> > > Thoughts?
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Sun, Aug 30, 2015 at 9:49 PM, Jinfeng Ni <jinfengni99@gmail.com>
> > wrote:
> > >
> > > > You are exactly right. We have 26 patches. Here they are
> > > > (--pretty=format:"%an %h %s")
> > > >
> > > > Jinfeng Ni 857ee95 VolcanoPlanner.clear() should clear ruleNames, in
> > > order
> > > > to avoid rule naming conflict error.
> > > > Jinfeng Ni ba2166f Make CalciteSchema extendable for different
> > > > implemetations by refactoring CalciteSchema and making it an
> interface
> > > > Hsuan-Yi Chu 046302d [CALCITE-827] Add a LogicalProject on top of
> > > > LogicalWindow to permute the columns in the same order as specified
> in
> > > the
> > > > SELECT list
> > > > Jinfeng Ni 2e076e1 Drill-specific change: Add back AbstractConverter
> in
> > > > RelSet.java and make sure AbstractConverter is not created when
> subset
> > > has
> > > > "NONE" conersion.
> > > > Mehant Baid 7ae335c Remove redundant invocation to ensureType in
> > > > RexBuilder.makeOver(). Don't invoke ensureType() for rewriting
> > aggregate
> > > > expressions part of window functions.
> > > > Jinfeng Ni 3bcce80 CALCITE-628 related but not fix theproblem: Ensure
> > > > target traits are simple when use Frameworks or RelOptRule.convert()
> > > > method.
> > > > vkorukanti e1876f0 Add new method to ViewExpander interface to allow
> > > > passing SchemaRoot.
> > > > Jinfeng Ni 8c8acee Match Logical Rel in ReduceExpressionRule.
> > > > Jason Altekruse a29f007 Enable inheriting from previously singleton
> > > filter
> > > > and calc constant reduction rules.
> > > > Jason Altekruse 15d645f Make the consent expression Executor
> > configurable
> > > > in FrameworkConfig.
> > > > Jinfeng Ni f176aa5 Disable the nullability cast checking in Filter.
> > > > Jinfeng Ni d0e2690 Add a new field name Suggester, so that duplicate
> > > names
> > > > will be renamed using the same policy as 0.9.
> > > > Jinfeng Ni 84bae71 Return validated row type and validated SqlNode
> when
> > > > call Planner.validate().
> > > > Aman Sinha 034e6e9 Workaround for CALCITE-528: Use case-insensitive
> > > > comparison when creating output row type of a JoinRel.
> > > > Aman Sinha 1f2fb1c Don't use 'SumEmptyIsZero' (SUM0) window aggregate
> > > until
> > > > CALCITE-777 is fixed.
> > > > Jinfeng Ni fcacd35 Make public the constructor of
> > > > SqlSumEmptyIsZeroAggFunction,
> > > > Aman Sinha 13d6449 Make certain push project constructors protected
> > such
> > > > that derived classes can access them.
> > > > Aman Sinha aebbb34 DRILL-1455: Add return type-inference strategy for
> > > > arithmetic operators when one of the arguments is ANY type.
> > > > Mehant Baid 32accb4 Some changes related to Sql ANY type.
> > > > Jinfeng Ni e182e2c [StarColumn] Use case insensitive in function name
> > > > comparison, in order to fix Sql Validator Exception when group by a
> > > > function expression.
> > > > Jinfeng Ni b75b374 [StarColumn] Make sure expression in the select
> list
> > > is
> > > > expanded at most once.
> > > > Jinfeng Ni bca12a6 [StarColumn] Keep the original field names from
> > left,
> > > > when rewrite a IN/EXISTS/NOT IN /NOT EXISTS subquery.  This is
> required
> > > to
> > > > support * column in schema-less system.
> > > > Jinfeng Ni eba9342 [StarColumn] Fix issues related to SqlIdentifier
> for
> > > > star column.
> > > > Hsuan-Yi Chu 70fcbca [StarColumn] When group-by a column, projecting
> > on a
> > > > star which cannot be expanded at planning time, use ITEM operator to
> > wrap
> > > > this column
> > > > Jinfeng Ni b3dc5b5 [StarColumn] Reverse one change in CALCITE-356,
> > which
> > > > regresses AggChecker logic, after * query in schema-less table is
> > added.
> > > > Jinfeng Ni 05fea03 [StarColumn] Support select * from schema-less
> table
> > > in
> > > > execution engine like Drill. Include support for view, subquery, CTE.
> > > >
> > > > I have pushed at least 2 patches to Calcite in the timeframe of
> > > > 1.4.0-release. While working on the rebasing work, I also try to wrap
> > > some
> > > > patches (including the "famous" calcite Schema change!) and make they
> > > ready
> > > > for pushing back to Calcite.
> > > >
> > > > In general, I agree that we should set a target to remove the forked
> > > > version. For the time frame of
> > > > 1.2 release, I will try to push at least 2 patches to Calcite
> master. I
> > > > also would like to have others
> > > > listed above spend sometime to push their corresponding patches to
> > > > Calcite.  For example, I talked
> > > > to Jason before, and he kindly agreed to push his two patches to
> > Calcite
> > > (
> > > > they are pretty ready to
> > > > merge to calcite master). There is also [CALCITE-827], which has been
> > > > already in Calcite next
> > > > release, but not in 1.4.0-release.
> > > >
> > > > However, I'm not sure if we could reduce the # of patches to 13 in
> 1.2,
> > > and
> > > > especially reduce to 2
> > > > by 1.3 release.
> > > >   - Many patches are to solve Drill's specific issues, for instance,
> > Star
> > > > column in schema-less
> > > >     system, AbstractConverter in VolcanoPlanner). That's why we even
> do
> > > not
> > > > create Calcite
> > > >     JIRA for them yet.  In other words, Calcite would work perfectly
> > > > without those patches, while
> > > >     Drill would fail lots of queries.
> > > >   - For some patches, we have tried to push to Calcite, but Calcite
> > > > community does not accept
> > > >    the patches. What should we do in such scenarios?
> > > >
> > > > I would welcome any help, if someone else would like to jump in and
> > offer
> > > > his/her help.
> > > >
> > > > Regards,
> > > >
> > > > Jinfeng
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Sun, Aug 30, 2015 at 4:59 PM, Jacques Nadeau <jacques@apache.org>
> > > > wrote:
> > > >
> > > > > Its great that we have rebased on top of Calcite 1.4.  However, it
> > > looks
> > > > > like we're still on a forked version of Calcite.  When I look here:
> > > > >
> https://github.com/mapr/incubator-calcite/commits/DrillCalcite1.4.0,
> > I
> > > > > count ~26 patches that still need to be merged (many of them
> without
> > > > even a
> > > > > Calcite bug id).
> > > > >
> > > > > I think we need to have a target number of non-merged patches or
> > we'll
> > > > > never get off the fork.  What do people think about bounding the
> > > release
> > > > on
> > > > > merging at least half of the items listed on that branch before we
> > > > release
> > > > > Drill 1.2 (~13) and we get down to no more than 2 for 1.3?
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message