drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Boaz Ben-Zvi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-7240) Run-time rowgroup pruning match() fails on casting a Long to an Integer
Date Sat, 04 May 2019 02:05:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832964#comment-16832964

Boaz Ben-Zvi commented on DRILL-7240:

Example recreating this bug - take this json file
{"key": "aa", "sales": 11}
{"key": "bb", "sales": 22}
And create two parquet tables/files by selecting from the json, first casting the "sales"
to an INT, and the second to a BIGINT:
create table test_int as select key, cast(sales as int) sales from dfs.`/tmp/myfile.json`;
create table test_bigint as select key, cast(sales as bigint) sales from dfs.`/tmp/myfile.json`;
Then move the two files into a sub-directory, renaming the second:
$ > mv /tmp/test_int/0_0_0.parquet /tmp/test/sub
$ > mv /tmp/test_bigint/0_0_0.parquet /tmp/test/sub/0_0_1.parquet 
Last refresh on only the first "key" columns, then run a query with a predicate on the 'sales"
refresh table METADATA columns(key) dfs.`/tmp/test`;
select sales from dfs.`/tmp/test/` where sales > 10;

> Run-time rowgroup pruning match() fails on casting a Long to an Integer
> -----------------------------------------------------------------------
>                 Key: DRILL-7240
>                 URL: https://issues.apache.org/jira/browse/DRILL-7240
>             Project: Apache Drill
>          Issue Type: Sub-task
>          Components: Storage - Parquet
>    Affects Versions: 1.17.0
>            Reporter: Boaz Ben-Zvi
>            Assignee: Boaz Ben-Zvi
>            Priority: Major
>             Fix For: 1.17.0
> After a Parquet table is refreshed with selected "interesting" columns, a query whose
WHERE clause contains a condition on a "non interesting" INT64 column fails during run-time
pruning (calling match()) with:
> {noformat}
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: ClassCastException: java.lang.Long
cannot be cast to java.lang.Integer
> {noformat}
>  Near-term fix suggestion: Catch the match() exception error, and instead do not prune
(i.e. run-time pruning would be disabled in such cases).

This message was sent by Atlassian JIRA

View raw message