hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <>
Subject [jira] [Commented] (HIVE-5302) PartitionPruner fails on Avro non-partitioned data
Date Sat, 28 Sep 2013 15:42:02 GMT


Edward Capriolo commented on HIVE-5302:

We do not necessarily need a documented testable case in the to justify the change, seeing
a non fatal error in the logs is reason enough to apply the patch.

In the matter of query plan bloat, we could isolate related changes to the Avro Serde so long
as there's a way to get at table properties during SerDe initialization. That way it could
check partition-specific and then fall back to table on its own. I'll worry about that once
I find a test case.
I would focus less on finding a test case. We can treat this as an optimization, and take
your word that their are cases where the current system does not work. See if you can find
this other way to solve this without effecting the plan, i think that is a big win for all
parties, if it is not possible there is nothing wrong with committing your original patch
in my eyes.
> PartitionPruner fails on Avro non-partitioned data
> --------------------------------------------------
>                 Key: HIVE-5302
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>    Affects Versions: 0.11.0
>            Reporter: Sean Busbey
>            Assignee: Sean Busbey
>            Priority: Blocker
>              Labels: avro
>         Attachments: HIVE-5302.1-branch-0.12.patch.txt, HIVE-5302.1.patch.txt, HIVE-5302.1.patch.txt
> While updating HIVE-3585 I found a test case that causes the failure in the MetaStoreUtils
partition retrieval from back in HIVE-4789.
> in this case, the failure is triggered when the partition pruner is handed a non-partitioned
table and has to construct a pseudo-partition.
> e.g.
> {code}
>   INSERT OVERWRITE TABLE partitioned_table PARTITION(col) SELECT id, foo, col FROM non_partitioned_table
WHERE col <= 9;
> {code}

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message