hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anthony Hsu" <>
Subject Re: Review Request 20096: HIVE-6835: Reading of partitioned Avro data fails if partition schema does not match table schema
Date Mon, 14 Apr 2014 18:49:55 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated April 14, 2014, 6:49 p.m.)

Review request for hive.


Addressed Carl's comments. Changes:
- Reverted whitespace changes.
- Moved the TABLE_PROP_PREFIX ("table.") to serdeConstants.
- Removed code that mutated the Properties passed to the AvroSerDe
- Added/improved comments
- Synced with latest

Repository: hive-git


The problem occurs when you store the "avro.schema.(literal|url)" in the SERDEPROPERTIES instead
of the TBLPROPERTIES, add a partition, change the table's schema, and then try reading from
the old partition.

I fixed this problem by passing the table properties to the partition with a "table." prefix,
and changing the Avro SerDe to always use the table properties when available.

Diffs (updated)

  ql/src/java/org/apache/hadoop/hive/ql/plan/ 43cef5c 
  ql/src/test/queries/clientpositive/avro_partitioned.q 6fe5117 
  ql/src/test/results/clientpositive/avro_partitioned.q.out 644716d 
  serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/ 515cf25

  serde/src/java/org/apache/hadoop/hive/serde2/avro/ 9d58d13 
  serde/src/test/org/apache/hadoop/hive/serde2/avro/ 67d5570 



Added test cases


Anthony Hsu

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message