drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3746) Hive query fails if the table contains external partitions
Date Thu, 10 Sep 2015 00:49:45 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737903#comment-14737903
] 

ASF GitHub Bot commented on DRILL-3746:
---------------------------------------

GitHub user vkorukanti opened a pull request:

    https://github.com/apache/drill/pull/151

    DRILL-3746: Get Hive partition values from MetaStore instead of from …

    …parsing the partition location path
    
    1) Added a partition with custom location to test Hive table. Existing partition tests
now work after the fix.
    2) Enabled a test which was disabled previously due to a bug in interpreter code
    
    @amansinha100 Could you review the patch?

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/vkorukanti/drill DRILL-3764

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/drill/pull/151.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #151
    
----
commit 90169e8ce7f46047631b25f5d248ffdf35788cbf
Author: vkorukanti <venki.korukanti@gmail.com>
Date:   2015-09-10T00:42:45Z

    DRILL-3746: Get Hive partition values from MetaStore instead of from parsing the partition
location path
    
    1) Added a partition with custom location to test Hive table. Existing partition tests
now work after the fix.
    2) Enabled a test which was disabled previously due to a bug in interpreter code

----


> Hive query fails if the table contains external partitions
> ----------------------------------------------------------
>
>                 Key: DRILL-3746
>                 URL: https://issues.apache.org/jira/browse/DRILL-3746
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Venki Korukanti
>
> If Hive contains a table which has external partitions, Drill fails in partition pruning
code, which causes the query to fail.
> {code}
> CREATE TABLE external_partition_test (boolean_field BOOLEAN) PARTITIONED BY (boolean_part
BOOLEAN);
> ALTER TABLE external_partition_test ADD PARTITION (boolean_part='true') LOCATION '/some/path';
> ALTER TABLE external_partition_test ADD PARTITION (boolean_part='false') LOCATION '/some/path';
> {code}
> Query:
> {code}
> SELECT * FROM hive.`default`.external_partition_test where boolean_part = false
> {code}
> Exception:
> {code}
> java.lang.StringIndexOutOfBoundsException
> String index out of range: -14
> at java.lang.String.substring(String.java:1875) ~[na:1.7.0_45]
> at org.apache.drill.exec.planner.sql.HivePartitionLocation.<init>(HivePartitionLocation.java:31) ~[classes/:na]
> at org.apache.drill.exec.planner.sql.HivePartitionDescriptor.getPartitions(HivePartitionDescriptor.java:117) ~[classes/:na]
> at org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:185) ~[classes/:na]
> at org.apache.drill.exec.planner.sql.logical.HivePushPartitionFilterIntoScan$2.onMatch(HivePushPartitionFilterIntoScan.java:92) ~[classes/:na]
> at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228) ~[calcite-core-1.4.0-drill-r0.jar:1.4.0-drill-r0]
> {code}
> Looking at {{HivePartitionLocation}}, it looks like we are depending on the organization
of files on FileSystem to get the partition values. We should get the partition values from
MetaStore.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message