Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Tue, 31 Oct 2017 19:13:00 +0000 (UTC)
From: "Vitalii Diravka (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.13105563.1506545229000.114470.1509477180335@Atlassian.JIRA>
In-Reply-To: <JIRA.13105563.1506545229000@Atlassian.JIRA>
References: <JIRA.13105563.1506545229000@Atlassian.JIRA> <JIRA.13105563.1506545229587@jira-lw-us.apache.org>
Subject: [jira] [Commented] (DRILL-5822) The query with "SELECT *" with
 "ORDER BY" clause and `planner.slice_target`=1 doesn't preserve column
 order
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 31 Oct 2017 19:13:05 -0000


    [ https://issues.apache.org/jira/browse/DRILL-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16227349#comment-16227349 ] 

Vitalii Diravka commented on DRILL-5822:
----------------------------------------

[~Paul.Rogers] 
Here is the issue of unnecessary sorting of columns for query with the following conditions: using wildcard in the query and ORDER BY clause, and when this is planned into multiple fragments ("alter session set `planner.slice_target`=1;").

The issue is connected to adding canonicalizing the schemas of input batches for Merging Receiver in DRILL-847. But this approach is outdated since for now in the process of loading batches in the RecordBatchLoader the new batch with same columns (SchemaPaths) but other ordering of them is perceived with same schema as the previous batch has: 
[All fields from the last batch is a hashMap structure|https://github.com/apache/drill/blob/fe79a633a3da8b4f6db50454fde64c30c73233bb/exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java#L90] and [when new batch appears the columns are just removed from the old one by the key|https://github.com/apache/drill/blob/fe79a633a3da8b4f6db50454fde64c30c73233bb/exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java#L102].
So the schemaChange flag still equals to false. And then [the schema will built|https://github.com/apache/drill/blob/fe79a633a3da8b4f6db50454fde64c30c73233bb/exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java#L138]. 

Here is only the issue that RecordBatchLoader permutes column order for the above case. And it was described in the jira ticket created by you DRILL-5828 and can be fixed there.
So my changes fix the current issue but not fully cover the requirements from your comment. Will It be reasonably if that changes will be done in context of DRILL-5828?

> The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1 doesn't preserve column order
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-5822
>                 URL: https://issues.apache.org/jira/browse/DRILL-5822
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.11.0
>            Reporter: Prasad Nagaraj Subramanya
>            Assignee: Vitalii Diravka
>             Fix For: 1.12.0
>
>
> Columns ordering doesn't preserve for the star query with sorting when this is planned into multiple fragments.
> Repro steps:
> 1) {code}alter session set `planner.slice_target`=1;{code}
> 2) ORDER BY clause in the query.
> Scenarios:
> {code}
> 0: jdbc:drill:zk=local> alter session reset `planner.slice_target`;
> +-------+--------------------------------+
> |  ok   |            summary             |
> +-------+--------------------------------+
> | true  | planner.slice_target updated.  |
> +-------+--------------------------------+
> 1 row selected (0.082 seconds)
> 0: jdbc:drill:zk=local> select * from cp.`tpch/nation.parquet` order by n_name limit 1;
> +--------------+----------+--------------+------------------------------------------------------+
> | n_nationkey  |  n_name  | n_regionkey  |                      n_comment                       |
> +--------------+----------+--------------+------------------------------------------------------+
> | 0            | ALGERIA  | 0            |  haggle. carefully final deposits detect slyly agai  |
> +--------------+----------+--------------+------------------------------------------------------+
> 1 row selected (0.141 seconds)
> 0: jdbc:drill:zk=local> alter session set `planner.slice_target`=1;
> +-------+--------------------------------+
> |  ok   |            summary             |
> +-------+--------------------------------+
> | true  | planner.slice_target updated.  |
> +-------+--------------------------------+
> 1 row selected (0.091 seconds)
> 0: jdbc:drill:zk=local> select * from cp.`tpch/nation.parquet` order by n_name limit 1;
> +------------------------------------------------------+----------+--------------+--------------+
> |                      n_comment                       |  n_name  | n_nationkey  | n_regionkey  |
> +------------------------------------------------------+----------+--------------+--------------+
> |  haggle. carefully final deposits detect slyly agai  | ALGERIA  | 0            | 0            |
> +------------------------------------------------------+----------+--------------+--------------+
> 1 row selected (0.201 seconds)
> {code}


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)