drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3993) Rebase Drill on Calcite master branch
Date Fri, 08 Dec 2017 17:20:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283867#comment-16283867
] 

ASF GitHub Bot commented on DRILL-3993:
---------------------------------------

GitHub user KulykRoman opened a pull request:

    https://github.com/apache/drill/pull/1066

    DRILL-3993: Changes to support Calcite 1.13

    Works with Drill-specific Calcite 1.13 from a branch: https://github.com/KulykRoman/incubator-calcite/tree/DRILL-3993.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/KulykRoman/drill DRILL-3993

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/drill/pull/1066.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1066
    
----
commit 71d8b03d60dcbe4d46f1f12aba82354064c25ef6
Author: Roman Kulyk <rom.kulyk@gmail.com>
Date:   2017-11-30T16:19:12Z

    REVERTED:  DRILL-5089
    
    Dynamically load schema of storage plugin only when needed for every query

commit 1c3cea13c798eb831f9e9911ec144bf4824ac5ad
Author: Roman Kulyk <rom.kulyk@gmail.com>
Date:   2017-08-29T14:10:24Z

    DRILL-3993: Changes to support Calcite 1.13
    
    - fixed all compiling errors (main changes were: Maven changes, chenges RelNode ->
RelRoot, implementing some new methods from updated interfaces, chenges some literals, logger
changes);
    - fixed unexpected column errors, validation errors and assertion errors after Calcite
update;
    - fixed describe table/schema statement according to updated logic;
    - added fixes with time-intervals;
    - changed precision of BINARY to 65536 (was 1048576) according to updated logic (Calcite
overrides bigger precision to own maxPrecision);
    - ignored some incorrect tests with DRILL-3244;
    - changed "Table not found" message to "Object not found within" according to new Calcite
changes.

commit 7196b366994f6ebcdcb80ca07701e06955e2b856
Author: Volodymyr Vysotskyi <vvovyk@gmail.com>
Date:   2017-11-03T12:18:09Z

    DRILL-3993: Fix unit test failures connected with support Calcite 1.13
    
    - Use root schema as default for describe table statement.
    Fix TestOpenTSDBPlugin.testDescribe() and TestInfoSchemaOnHiveStorage.varCharMaxLengthAndDecimalPrecisionInInfoSchema()
unit tests.
    - Modify expected results for tests:
    TestPreparedStatementProvider.invalidQueryValidationError();
    TestProjectPushDown.testTPCH1();
    TestProjectPushDown.testTPCH3();
    TestStorageBasedHiveAuthorization.selectUser1_db_u0_only();
    TestStorageBasedHiveAuthorization.selectUser0_db_u1g1_only()
    - Fix TestCTAS.whenTableQueryColumnHasStarAndTableFiledListIsSpecified(), TestViewSupport.createViewWhenViewQueryColumnHasStarAndViewFiledListIsSpecified(),
TestInbuiltHiveUDFs.testIf(), testDisableUtf8SupportInQueryString unit tests.
    - Fix UnsupportedOperationException and NPE for jdbc tests.
    - Fix AssertionError: Conversion to relational algebra failed to preserve datatypes
    
    *DrillCompoundIdentifier:
    According to the changes, made in [CALCITE-546], star Identifier is replaced by empty
string during parsing the query. Since Drill uses its own DrillCompoundIdentifier, it should
also replace star by empty string before creating SqlIdentifier instance to avoid further
errors connected with star column. see SqlIdentifier.isStar() method.
    
    *SqlConverter:
    In [CALCITE-1417] added simplification of expressions which should be projected every
time when a new project rel node is created using RelBuilder. It causes assertion errors connected
with types nullability. This hook was set to false to avoid project expressions simplification.
See usage of this hook and RelBuilder.project() method.
    
    In Drill the type nullability of the function depends on only the nullability of its arguments.
In some cases, a function may return null value even if it had non-nullable arguments. When
Calice simplifies expressions, it checks that the type of the result is the same as the type
of the expression. Otherwise, makeCast() method is called. But when a function returns null
literal, this cast does nothing, even when the function has a non-nullable type. So to avoid
this issue, method makeCast() was overridden.
    
    *DrillAvgVarianceConvertlet:
    Problem with sum0 and specific changes in old Calcite (it is CALCITE-777). (see HistogramShuttle.visitCall
method) Changes were made to avoid changes in Calcite.
    
    *SqlConverter, DescribeTableHandler, ShowTablesHandler:
    New Calcite tries to combine both default and specified workspaces during the query validation.
In some cases, for example, when describe table statement is used, Calcite tries to find INFORMATION_SCHEMA
in the schema used as default. When it does not find the schema, it tries to find a table
with such name. For some storage plugins, such as opentsdb and hbase, when a table was not
found, the error is thrown, and the query fails. To avoid this issue, default schema was changed
to root schema for validation stage for describe table and show tables queries.

commit 56f2c9ac47d224e2ac02ab5036e7aa01df1951f2
Author: Volodymyr Vysotskyi <vvovyk@gmail.com>
Date:   2017-11-15T10:37:50Z

    DRILL-3993: Use custom RelBuilder implementation in rules
    
    After the changes, made in CALCITE-1056 if the filter has a predicate that is always false,
RelBuilder.filter() method returns values rel node instead of filter rel node. In order to
preserve column types, DrillRelBuilder.empty() method, which is returned by filter method
 was overridden, and now it returns filter with a false predicate. (advice to override this
method was in its javadoc) The goal of all other changes in this commit is to use our custom
RelBuilder for all rules that are used in Drill.

commit 82c750aee2ad67a1e9e0227ca06d39f9362cba99
Author: Roman Kulyk <rom.kulyk@gmail.com>
Date:   2017-11-02T18:22:36Z

    DRILL-3993: Fix failed tests after Calcite update
    
    - fix temporary table errors according to updated logic;
    - fixed errors when we trying to make select from hbase table with schema name in query
(example: "SELECT row_key FROM hbase.TestTableNullStr) from hbase schema (did "USE hbase"
before). Added test for it;
    - added fix for views which were created on Calcite 1.4 and test for it.

----


> Rebase Drill on Calcite master branch
> -------------------------------------
>
>                 Key: DRILL-3993
>                 URL: https://issues.apache.org/jira/browse/DRILL-3993
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 1.2.0
>            Reporter: Sudheesh Katkam
>            Assignee: Roman Kulyk
>
> Calcite keeps moving, and now we need to catch up to Calcite 1.5, and ensure there are
no regressions.
> Also, how do we resolve this 'catching up' issue in the long term?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message