drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5691) multiple count distinct query planning error at physical phase
Date Tue, 08 Aug 2017 10:27:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118148#comment-16118148

ASF GitHub Bot commented on DRILL-5691:

Github user arina-ielchiieva commented on the issue:

    @weijietong regarding the unit test, I have tried to reproduce the problem and written
the following unit test:
      public void test() throws Exception {
        FileSystem fs = null;
        try {
          fs = FileSystem.get(new Configuration());
          // create table with partition pruning
          test("use dfs_test.tmp");
          String tableName = "table_with_pruning";
          Path dataFile = new Path(TestTools.getWorkingPath(),"src/test/resources/parquet/alltypes_required.parquet");
          test("create table %s partition by (col_int) as select * from dfs.`%s`", tableName,
          // generate metadata
          test("refresh table metadata `%s`", tableName);
          // execute query
          String query = String.format("select count(distinct col_int), count(distinct col_chr)
from `%s` where col_int = 45436", tableName);
        } finally {
          if (fs != null) {
    `AbstractGroupScan.getScanStats` method returns one row but it does not fail. Can you
please take a look?

> multiple count distinct query planning error at physical phase 
> ---------------------------------------------------------------
>                 Key: DRILL-5691
>                 URL: https://issues.apache.org/jira/browse/DRILL-5691
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Relational Operators
>    Affects Versions: 1.9.0, 1.10.0
>            Reporter: weijie.tong
> I materialized the count distinct query result in a cache , added a plugin rule to translate
the (Aggregate、Aggregate、Project、Scan) or (Aggregate、Aggregate、Scan) to (Project、Scan)
at the PARTITION_PRUNING phase. Then ,once user issue count distinct queries , it will be
translated to query the cache to get the result.
> eg1: " select count(*),sum(a) ,count(distinct b)  from t where dt=xx " 
> eg2:"select count(*),sum(a) ,count(distinct b) ,count(distinct c) from t where dt=xxx
> eg3:"select count(distinct b), count(distinct c) from t where dt=xxx"
> eg1 will be right and have a query result as I expected , but eg2 will be wrong at the
physical phase.The error info is here: https://gist.github.com/weijietong/1b8ed12db9490bf006e8b3fe0ee52269.

> eg3 will also get the similar error.

This message was sent by Atlassian JIRA

View raw message