hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho" <sze...@cloudera.com>
Subject Review Request 18925: HIVE-6575 select * fails on parquet table with map datatype
Date Fri, 07 Mar 2014 22:12:23 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18925/
-----------------------------------------------------------

Review request for hive, Brock Noland, justin coffey, and Xuefu Zhang.


Repository: hive-git


Description
-------

The issue is, as part of select * query, a DeepParquetHiveMapInspector is used for one column
of an overall parquet-table struct object inspector.  

The problem lies in the ObjectInspectorFactory's cache for struct object inspector.  For performance
it caches based on an array list, of all object inspectors of columns.  The second time the
query is run, it is attempted to lookup cached value.  When the DeepParquetHiveMapInspector
is gotten to, java calls .equals but it fails as the .equals method casted it to a "StandardParquetHiveInspector".

Regenerating the .equals and .hashcode from eclipse.  

Also adding one more check in .equals before casting, as if by luck another type of object
inspector gets hashed to the same hashcode in the cache, java would call .equals against it
to find it if it is the key.


Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/AbstractParquetMapInspector.java
1d72747 

Diff: https://reviews.apache.org/r/18925/diff/


Testing
-------

Manual testing.


Thanks,

Szehon Ho


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message