hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho" <sze...@cloudera.com>
Subject Re: Review Request 18925: HIVE-6575 select * fails on parquet table with map datatype
Date Sat, 08 Mar 2014 00:01:16 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18925/
-----------------------------------------------------------

(Updated March 8, 2014, 12:01 a.m.)


Review request for hive, Brock Noland, justin coffey, and Xuefu Zhang.


Changes
-------

Fixed the styling of generated equals code.  I looked at the proposed refactor, I think the
simplication you suggested would hit NPE if this.keyInspector is null and other.keyInspector
!= null.  Given that you would need a few more boolean statements in that line to make it
correct, it seems that the current one is probably the cleanest to look at.


Repository: hive-git


Description
-------

The issue is, as part of select * query, a DeepParquetHiveMapInspector is used for one column
of an overall parquet-table struct object inspector.  

The problem lies in the ObjectInspectorFactory's cache for struct object inspector.  For performance,
there is a cache keyed on an array list, of all object inspectors of columns.  The second
time the query is run, it attempts to lookup cached struct inspector.  But when the hashmap
looks up the part of the key consisting of the DeepParquetHiveMapInspector, java calls .equals
against the existing DeepParquetHivemapInspector.  This fails, as the .equals method casted
the "other" to a "StandardParquetHiveInspector".

Regenerating the .equals and .hashcode from eclipse.  

Also adding one more check in .equals before casting, to handle the case if another class
of object inspector gets hashed to the same hashcode in the cache.  Then java would call .equals
against the other, which in this case is not of the same class.


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/AbstractParquetMapInspector.java
1d72747 

Diff: https://reviews.apache.org/r/18925/diff/


Testing
-------

Manual testing.


Thanks,

Szehon Ho


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message