drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jinfeng Ni <jinfengn...@gmail.com>
Subject Re: Unforking Calcite...
Date Mon, 31 Aug 2015 04:49:54 GMT
You are exactly right. We have 26 patches. Here they are
(--pretty=format:"%an %h %s")

Jinfeng Ni 857ee95 VolcanoPlanner.clear() should clear ruleNames, in order
to avoid rule naming conflict error.
Jinfeng Ni ba2166f Make CalciteSchema extendable for different
implemetations by refactoring CalciteSchema and making it an interface
Hsuan-Yi Chu 046302d [CALCITE-827] Add a LogicalProject on top of
LogicalWindow to permute the columns in the same order as specified in the
SELECT list
Jinfeng Ni 2e076e1 Drill-specific change: Add back AbstractConverter in
RelSet.java and make sure AbstractConverter is not created when subset has
"NONE" conersion.
Mehant Baid 7ae335c Remove redundant invocation to ensureType in
RexBuilder.makeOver(). Don't invoke ensureType() for rewriting aggregate
expressions part of window functions.
Jinfeng Ni 3bcce80 CALCITE-628 related but not fix theproblem: Ensure
target traits are simple when use Frameworks or RelOptRule.convert() method.
vkorukanti e1876f0 Add new method to ViewExpander interface to allow
passing SchemaRoot.
Jinfeng Ni 8c8acee Match Logical Rel in ReduceExpressionRule.
Jason Altekruse a29f007 Enable inheriting from previously singleton filter
and calc constant reduction rules.
Jason Altekruse 15d645f Make the consent expression Executor configurable
in FrameworkConfig.
Jinfeng Ni f176aa5 Disable the nullability cast checking in Filter.
Jinfeng Ni d0e2690 Add a new field name Suggester, so that duplicate names
will be renamed using the same policy as 0.9.
Jinfeng Ni 84bae71 Return validated row type and validated SqlNode when
call Planner.validate().
Aman Sinha 034e6e9 Workaround for CALCITE-528: Use case-insensitive
comparison when creating output row type of a JoinRel.
Aman Sinha 1f2fb1c Don't use 'SumEmptyIsZero' (SUM0) window aggregate until
CALCITE-777 is fixed.
Jinfeng Ni fcacd35 Make public the constructor of
SqlSumEmptyIsZeroAggFunction,
Aman Sinha 13d6449 Make certain push project constructors protected such
that derived classes can access them.
Aman Sinha aebbb34 DRILL-1455: Add return type-inference strategy for
arithmetic operators when one of the arguments is ANY type.
Mehant Baid 32accb4 Some changes related to Sql ANY type.
Jinfeng Ni e182e2c [StarColumn] Use case insensitive in function name
comparison, in order to fix Sql Validator Exception when group by a
function expression.
Jinfeng Ni b75b374 [StarColumn] Make sure expression in the select list is
expanded at most once.
Jinfeng Ni bca12a6 [StarColumn] Keep the original field names from left,
when rewrite a IN/EXISTS/NOT IN /NOT EXISTS subquery.  This is required to
support * column in schema-less system.
Jinfeng Ni eba9342 [StarColumn] Fix issues related to SqlIdentifier for
star column.
Hsuan-Yi Chu 70fcbca [StarColumn] When group-by a column, projecting on a
star which cannot be expanded at planning time, use ITEM operator to wrap
this column
Jinfeng Ni b3dc5b5 [StarColumn] Reverse one change in CALCITE-356, which
regresses AggChecker logic, after * query in schema-less table is added.
Jinfeng Ni 05fea03 [StarColumn] Support select * from schema-less table in
execution engine like Drill. Include support for view, subquery, CTE.

I have pushed at least 2 patches to Calcite in the timeframe of
1.4.0-release. While working on the rebasing work, I also try to wrap some
patches (including the "famous" calcite Schema change!) and make they ready
for pushing back to Calcite.

In general, I agree that we should set a target to remove the forked
version. For the time frame of
1.2 release, I will try to push at least 2 patches to Calcite master. I
also would like to have others
listed above spend sometime to push their corresponding patches to
Calcite.  For example, I talked
to Jason before, and he kindly agreed to push his two patches to Calcite (
they are pretty ready to
merge to calcite master). There is also [CALCITE-827], which has been
already in Calcite next
release, but not in 1.4.0-release.

However, I'm not sure if we could reduce the # of patches to 13 in 1.2, and
especially reduce to 2
by 1.3 release.
  - Many patches are to solve Drill's specific issues, for instance, Star
column in schema-less
    system, AbstractConverter in VolcanoPlanner). That's why we even do not
create Calcite
    JIRA for them yet.  In other words, Calcite would work perfectly
without those patches, while
    Drill would fail lots of queries.
  - For some patches, we have tried to push to Calcite, but Calcite
community does not accept
   the patches. What should we do in such scenarios?

I would welcome any help, if someone else would like to jump in and offer
his/her help.

Regards,

Jinfeng





On Sun, Aug 30, 2015 at 4:59 PM, Jacques Nadeau <jacques@apache.org> wrote:

> Its great that we have rebased on top of Calcite 1.4.  However, it looks
> like we're still on a forked version of Calcite.  When I look here:
> https://github.com/mapr/incubator-calcite/commits/DrillCalcite1.4.0, I
> count ~26 patches that still need to be merged (many of them without even a
> Calcite bug id).
>
> I think we need to have a target number of non-merged patches or we'll
> never get off the fork.  What do people think about bounding the release on
> merging at least half of the items listed on that branch before we release
> Drill 1.2 (~13) and we get down to no more than 2 for 1.3?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message