hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dere (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-5356) Move arithmatic UDFs to generic UDF implementations
Date Mon, 04 Nov 2013 23:06:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13813398#comment-13813398
] 

Jason Dere commented on HIVE-5356:
----------------------------------

Just want to point out to folks that this does result in some changes in return type behavior
for arithmetic operators, some of which were a bit of a surprise:

int / int => decimal
float / float => double
float * float => double
float + float => double

That said, this is still in line with what the SQL standard says about the resulting type
of these arithmetic operators (shown below), so I think this is ok:

1) If the declared type of both operands of a dyadic arithmetic operator is exact numeric,
then the declared type of the result is an implementation-defined exact numeric type, with
precision and scale determined as follows:
a) Let S1 and S2 be the scale of the first and second operands respectively.
b) The precision of the result of addition and subtraction is implementation-defined, and
the scale is the
maximum of S1 and S2.
c) The precision of the result of multiplication is implementation-defined, and the scale
is S1 + S2.
d) The precision and scale of the result of division are implementation-defined.
2) If the declared type of either operand of a dyadic arithmetic operator is approximate numeric,
then the declared type of the result is an implementation-defined approximate numeric type.

> Move arithmatic UDFs to generic UDF implementations
> ---------------------------------------------------
>
>                 Key: HIVE-5356
>                 URL: https://issues.apache.org/jira/browse/HIVE-5356
>             Project: Hive
>          Issue Type: Task
>          Components: UDF
>    Affects Versions: 0.11.0
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>             Fix For: 0.13.0
>
>         Attachments: HIVE-5356.1.patch, HIVE-5356.2.patch, HIVE-5356.patch
>
>
> Currently, all of the arithmetic operators, such as add/sub/mult/div, are implemented
as old-style UDFs and java reflection is used to determine the return type TypeInfos/ObjectInspectors,
based on the return type of the evaluate() method chosen for the expression. This works fine
for types that don't have type params.
> Hive decimal type participates in these operations just like int or double. Different
from double or int, however, decimal has precision and scale, which cannot be determined by
just looking at the return type (decimal) of the UDF evaluate() method, even though the operands
have certain precision/scale. With the default of "decimal" without precision/scale, then
(10, 0) will be the type params. This is certainly not desirable.
> To solve this problem, all of the arithmetic operators would need to be implemented as
GenericUDFs, which allow returning ObjectInspector during the initialize() method. The object
inspectors returned can carry type params, from which the "exact" return type can be determined.
> It's worth mentioning that, for user UDF implemented in non-generic way, if the return
type of the chosen evaluate() method is decimal, the return type actually has (10,0) as precision/scale,
which might not be desirable. This needs to be documented.
> This JIRA will cover minus, plus, divide, multiply, mod, and pmod, to limit the scope
of review. The remaining ones will be covered under HIVE-5706.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message