hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Wilfong" <kevinwilf...@fb.com>
Subject Review Request: Warn user that precision is lost when bigint is implicitly cast to double in joins and unions.
Date Fri, 02 Sep 2011 16:56:47 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1705/
-----------------------------------------------------------

Review request for hive and Siying Dong.


Summary
-------

I added checks before the type conversions for both joins and unions to either throw an error
or a warning depending if a bigint was going to be converted to a double.

I also added a new variable hive.mapred.bigint.comparison.mode, which is set to either strict
or nonstrict to indicate whether an error or a warning should be given when a bigint is converted
to a double.  This is instead of the original implementation which used the variable hive.mapred.mode.


This addresses bug HIVE-2427.
    https://issues.apache.org/jira/browse/HIVE-2427


Diffs
-----

  trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1164293 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1164293 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1164293 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java 1164293 
  trunk/ql/src/test/queries/clientnegative/compare_double_bigint.q 1164293 
  trunk/ql/src/test/queries/clientnegative/compare_double_bigint_join.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/compare_double_bigint_union.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/compare_string_bigint.q 1164293 
  trunk/ql/src/test/queries/clientnegative/compare_string_bigint_join.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/compare_string_bigint_union.q PRE-CREATION 
  trunk/ql/src/test/queries/clientpositive/filter_join_breaktask2.q 1164293 
  trunk/ql/src/test/results/clientnegative/compare_double_bigint.q.out 1164293 
  trunk/ql/src/test/results/clientnegative/compare_double_bigint_join.q.out PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/compare_double_bigint_union.q.out PRE-CREATION

  trunk/ql/src/test/results/clientnegative/compare_string_bigint.q.out 1164293 
  trunk/ql/src/test/results/clientnegative/compare_string_bigint_join.q.out PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/compare_string_bigint_union.q.out PRE-CREATION

  trunk/ql/src/test/results/clientpositive/filter_join_breaktask2.q.out 1164293 

Diff: https://reviews.apache.org/r/1705/diff


Testing
-------

I added several more tests to demonstrate the errors.

I also modified a unit test in which a join was being done which involved a comparison between
a string and a bigint.  I set hive.mapred.bigint.comparison.mode to nonstrict, so the only
change was the new warning appeared in the output.


Thanks,

Kevin


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message