Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Thu, 12 Nov 2015 05:53:10 +0000 (UTC)
From: "Aman Sinha (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.12912056.1447208103000.47399.1447307590940@Atlassian.JIRA>
In-Reply-To: <JIRA.12912056.1447208103000@Atlassian.JIRA>
References: <JIRA.12912056.1447208103000@Atlassian.JIRA>
 <JIRA.12912056.1447208103231@arcas>
Subject: [jira] [Commented] (DRILL-4071) Partition pruning fails when a
 Coalesce() function appears with partition filter
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/DRILL-4071?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1500=
1732#comment-15001732 ]=20

Aman Sinha commented on DRILL-4071:
-----------------------------------

Code change looks good.  Tests need some cleanup: remove the hardcoded path=
 that has username.  Also, although the tests do check for number of files =
after pruning, they produce 0 rows...it would be better to have filters tha=
t produce non-zero result set.=20

> Partition pruning fails when a Coalesce() function appears with partition=
 filter
> -------------------------------------------------------------------------=
-------
>
>                 Key: DRILL-4071
>                 URL: https://issues.apache.org/jira/browse/DRILL-4071
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>            Reporter: Aman Sinha
>            Assignee: Aman Sinha
>         Attachments: DRILL-4071.patch
>
>
> Pruning fails for this query:=20
> {code}
> 0: jdbc:drill:zk=3Dlocal> explain plan for select count(*) from dfs.`/Use=
rs/asinha/data/multilevel/parquet` where dir0 =3D 1994 and coalesce(o_clerk=
, 'Clerk') =3D '';
> +------+------+
> | text | json |
> +------+------+
> | 00-00    Screen
> 00-01      Project(EXPR$0=3D[$0])
> 00-02        StreamAgg(group=3D[{}], EXPR$0=3D[COUNT()])
> 00-03          Project($f0=3D[0])
> 00-04            SelectionVectorRemover
> 00-05              Filter(condition=3D[AND(=3D($0, 1994), =3D(CASE(IS NOT=
 NULL($1), $1, 'Clerk'), ''))])
> 00-06                Scan(groupscan=3D[ParquetGroupScan [entries=3D[ReadE=
ntryWithPath [path=3Dfile:/Users/asinha/data/multilevel/parquet/1994/Q1/ord=
ers_94_q1.parquet], ReadEntryWithPath [path=3Dfile:/Users/asinha/data/multi=
level/parquet/1994/Q2/orders_94_q2.parquet], ReadEntryWithPath [path=3Dfile=
:/Users/asinha/data/multilevel/parquet/1994/Q3/orders_94_q3.parquet], ReadE=
ntryWithPath [path=3Dfile:/Users/asinha/data/multilevel/parquet/1994/Q4/ord=
ers_94_q4.parquet], ReadEntryWithPath [path=3Dfile:/Users/asinha/data/multi=
level/parquet/1995/Q1/orders_95_q1.parquet], ReadEntryWithPath [path=3Dfile=
:/Users/asinha/data/multilevel/parquet/1995/Q2/orders_95_q2.parquet], ReadE=
ntryWithPath [path=3Dfile:/Users/asinha/data/multilevel/parquet/1995/Q3/ord=
ers_95_q3.parquet], ReadEntryWithPath [path=3Dfile:/Users/asinha/data/multi=
level/parquet/1995/Q4/orders_95_q4.parquet], ReadEntryWithPath [path=3Dfile=
:/Users/asinha/data/multilevel/parquet/1996/Q1/orders_96_q1.parquet], ReadE=
ntryWithPath [path=3Dfile:/Users/asinha/data/multilevel/parquet/1996/Q2/ord=
ers_96_q2.parquet], ReadEntryWithPath [path=3Dfile:/Users/asinha/data/multi=
level/parquet/1996/Q3/orders_96_q3.parquet], ReadEntryWithPath [path=3Dfile=
:/Users/asinha/data/multilevel/parquet/1996/Q4/orders_96_q4.parquet]], sele=
ctionRoot=3Dfile:/Users/asinha/data/multilevel/parquet, numFiles=3D12, used=
MetadataFile=3Dfalse, columns=3D[`dir0`, `o_clerk`]]])
> {code}
> The log indicates no partition filters were found:=20
> {code}
> ...
> o.a.d.e.p.l.partition.PruneScanRule - No conditions were found eligible f=
or partition pruning.Total pruning elapsed time: 0 ms
> {code}
> A preliminary analysis indicates that since the Coalesce gets converted t=
o a CASE(IS NOT NULL....) expression, the filter analysis does not correctl=
y process the full expression tree.  At one point in {{FindPartitionConditi=
ons.analyzeCall()}} I saw the operandStack had 3 elements in it: [NO_PUSH, =
NO_PUSH, PUSH] which seemed strange since I would expect even number of ele=
ments.=20


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)