hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From " (Commented) (JIRA)" <>
Subject [jira] [Commented] (HIVE-2427) Warn user that precision is lost when bigint is implicitly cast to double in joins and unions.
Date Fri, 04 Nov 2011 19:49:53 GMT

] commented on HIVE-2427:

This is an automatically generated e-mail. To reply, visit:

(Updated 2011-11-04 19:48:37.718693)

Review request for hive, Yongqiang He, Ning Zhang, and Siying Dong.


It's been a while, so I ran svn up and fixed the conflicts.

I ran the tests and verified they passed.


I added checks before the type conversions for both joins and unions to either throw an error
or a warning depending if a bigint was going to be converted to a double.

I also added a new variable hive.mapred.bigint.comparison.mode, which is set to either strict
or nonstrict to indicate whether an error or a warning should be given when a bigint is converted
to a double.  This is instead of the original implementation which used the variable hive.mapred.mode.

This addresses bug HIVE-2427.

Diffs (updated)

  trunk/common/src/java/org/apache/hadoop/hive/conf/ 1197375 
  trunk/conf/hive-default.xml 1197375 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ 1197375 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ 1197375 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ 1197375 
  trunk/ql/src/test/queries/clientnegative/compare_double_bigint.q 1197375 
  trunk/ql/src/test/queries/clientnegative/compare_double_bigint_join.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/compare_double_bigint_union.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/compare_string_bigint.q 1197375 
  trunk/ql/src/test/queries/clientnegative/compare_string_bigint_join.q PRE-CREATION 
  trunk/ql/src/test/queries/clientnegative/compare_string_bigint_union.q PRE-CREATION 
  trunk/ql/src/test/queries/clientpositive/filter_join_breaktask2.q 1197375 
  trunk/ql/src/test/results/clientnegative/compare_double_bigint.q.out 1197375 
  trunk/ql/src/test/results/clientnegative/compare_double_bigint_join.q.out PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/compare_double_bigint_union.q.out PRE-CREATION

  trunk/ql/src/test/results/clientnegative/compare_string_bigint.q.out 1197375 
  trunk/ql/src/test/results/clientnegative/compare_string_bigint_join.q.out PRE-CREATION 
  trunk/ql/src/test/results/clientnegative/compare_string_bigint_union.q.out PRE-CREATION

  trunk/ql/src/test/results/clientpositive/filter_join_breaktask2.q.out 1197375 



I added several more tests to demonstrate the errors.

I also modified a unit test in which a join was being done which involved a comparison between
a string and a bigint.  I set hive.mapred.bigint.comparison.mode to nonstrict, so the only
change was the new warning appeared in the output.



> Warn user that precision is lost when bigint is implicitly cast to double in joins and
> ----------------------------------------------------------------------------------------------
>                 Key: HIVE-2427
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>         Attachments: HIVE-2427.1.patch.txt, HIVE-2427.2.patch.txt, HIVE-2427.3.patch.txt
> When a bigint is implicitly cast to a double (when a bigint is involved in an equality
expression with a string or double as part of a join condition, or when a bigint column is
to be combined with a string or double column as part of a union) precision may be lost, resulting
in unexpected behavior. Until we fix the underlying issue we should throw an error in strict
mode, and a warning in nonstrict mode alerting the user about this.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message