hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <>
Subject [jira] [Commented] (HIVE-5302) PartitionPruner fails on Avro non-partitioned data
Date Wed, 18 Sep 2013 17:12:52 GMT


Ashutosh Chauhan commented on HIVE-5302:

And as an extension to that all table level properties will now also automagically appear
as partition properties which doesn't feel right. Normally, it should never be a requirement
that partition need to know table properties. Problem arises because of weirdity in how AvroSerde
works since it stores its schema in properties object instead of in metastore columns table.
I think this problem is too specific to Avro, so this should be done in Avro specific code,
AvroSerde perhaps.
> PartitionPruner fails on Avro non-partitioned data
> --------------------------------------------------
>                 Key: HIVE-5302
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>    Affects Versions: 0.11.0
>            Reporter: Sean Busbey
>            Assignee: Sean Busbey
>            Priority: Blocker
>              Labels: avro
>         Attachments: HIVE-5302.1-branch-0.12.patch.txt, HIVE-5302.1.patch.txt, HIVE-5302.1.patch.txt
> While updating HIVE-3585 I found a test case that causes the failure in the MetaStoreUtils
partition retrieval from back in HIVE-4789.
> in this case, the failure is triggered when the partition pruner is handed a non-partitioned
table and has to construct a pseudo-partition.
> e.g.
> {code}
>   INSERT OVERWRITE TABLE partitioned_table PARTITION(col) SELECT id, foo, col FROM non_partitioned_table
WHERE col <= 9;
> {code}

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message