drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arina Ielchiieva (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (DRILL-5538) Exclude ProjectRemoveRule during PHYSICAL phase if it comes from storage plugins
Date Wed, 31 May 2017 16:32:04 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16031464#comment-16031464
] 

Arina Ielchiieva edited comment on DRILL-5538 at 5/31/17 4:31 PM:
------------------------------------------------------------------

[~jni] I have discussed this problem with Calcite team (you can see conversation in CALCITE-1584
Jira). Basically, Julian is saying that 
{quote}
Calcite doesn't promise to retain field names. 95% of the time it can and does, but 5% of
the time it really can't retain field names without seriously compromising the Volcano engine.
{quote}
Agree that removing this rule from Drill at first point might seems too rough decision. Regarding
impact on JDBC storage plugin, I am not sure what it can be, from the code I can't tell if
it needs this rule or not. I guess we might need to think of some other options here.
For example, will it make sense to separate alias assigning into separate stage from project
(scan -> project -> alias)? Not sure if it's possible though. And I guess it is an overhead
:)
One other option we might replace ProjectRemoveRule in JDBC plugin with DrillProjectRemoveRule.
It would be the same as in Calcite except of its isTrivial method would consider not only
row types but also column names when deciding if project is trivial or not. Actually, previously
in Calcite this method was considering column names too but this logic was removed.


was (Author: arina):
[~jni] I have discussed this problem with Calcite team (you can see conversation in CALCITE-1584
Jira). Basically, Julian is saying that 
{quote}
Calcite doesn't promise to retain field names. 95% of the time it can and does, but 5% of
the time it really can't retain field names without seriously compromising the Volcano engine.
{quote}
Agree that removing this rule from Drill at first point might seems too rough decision. Regarding
impact on JDBC storage plugin, I am not sure what it can be, from the code I can't tell if
it needs this rule or not. I guess we might need to think of some other options here.
For example, will it make sense to separate alias assigning into separate stage from project
(scan -> project -> alias)? Not sure if it's possible though. And I guess is an overhead
:)
One other option we might replace ProjectRemoveRule in JDBC plugin with DrillProjectRemoveRule.
It would be the same as in Calcite except of its isTrivial method would consider not only
row types but also column names when deciding if project is trivial or not. Actually, previously
in Calcite this method was considering column names too but this logic was removed.

> Exclude ProjectRemoveRule during PHYSICAL phase if it comes from storage plugins
> --------------------------------------------------------------------------------
>
>                 Key: DRILL-5538
>                 URL: https://issues.apache.org/jira/browse/DRILL-5538
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 1.10.0
>            Reporter: Arina Ielchiieva
>            Assignee: Arina Ielchiieva
>
> When [RDBMS storage plugin|https://drill.apache.org/docs/rdbms-storage-plugin/]  is enabled,
during query execution certain JDBC rules are added.
> One of the rules is [ProjectRemoveRule|https://github.com/apache/drill/blob/master/contrib/storage-jdbc/src/main/java/org/apache/drill/exec/store/jdbc/JdbcStoragePlugin.java#L140].
Drill also uses this rule but during phases when it considers it useful, for example, during
LOGICAL and JOIN_PLANNING. On the contrary, storage plugin rules are added to any phase of
query planning. Thus it results to project stage to be removed when actually it is needed.
> Sometimes when ProjectRemoveRule decides that project is trivial and removes it, during
this stage Drill added column alias or removed implicit columns.
> For example, with RDBMS plugin enabled, alias is not displayed for simple query:
> {noformat}
> 0: jdbc:drill:zk=local> create temporary table t as select * from sys.version;
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
> +-----------+----------------------------+
> | Fragment  | Number of records written  |
> +-----------+----------------------------+
> | 0_0       | 1                          |
> +-----------+----------------------------+
> 1 row selected (0.623 seconds)
> 0: jdbc:drill:zk=local> select version as current_version from t;
> +------------------+
> |     version      |
> +------------------+
> | 1.11.0-SNAPSHOT  |
> +------------------+
> 1 row selected (0.28 seconds)
> {noformat}
> Proposed fix is to exclude ProjectRemoveRule during PHYSICAL phase if it comes from storage
plugins to prevent Drill losing column alias or displaying implicit columns.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message