drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Gilmore (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-2517) Partition pruning still reading files during planning
Date Mon, 23 Mar 2015 06:26:10 GMT
Adam Gilmore created DRILL-2517:

             Summary: Partition pruning still reading files during planning
                 Key: DRILL-2517
                 URL: https://issues.apache.org/jira/browse/DRILL-2517
             Project: Apache Drill
          Issue Type: Bug
          Components: Query Planning & Optimization
    Affects Versions: 0.7.0, 0.8.0
            Reporter: Adam Gilmore
            Assignee: Jinfeng Ni

Partition pruning still tries to read Parquet files during the planning stage even though
they don't match the partition filter.

For example:

0: jdbc:drill:zk=local> select sum(price) from dfs.tmp.purchases where dir0 = 1;
Query failed: IllegalArgumentException: file:/tmp/purchases/4/0_0_0.parquet is not a Parquet
file (too small)

The reason is that the partition pruning happens after the Parquet plugin tries to read the
footer of each file.

Ideally, partition pruning would happen first before the format plugin gets involved.

This message was sent by Atlassian JIRA

View raw message