hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aihua Xu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-13423) Handle the overflow case for decimal datatype for sum()
Date Fri, 14 Oct 2016 21:17:20 GMT

     [ https://issues.apache.org/jira/browse/HIVE-13423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aihua Xu updated HIVE-13423:
----------------------------
    Description: 
When a column col1 defined as decimal and if the sum of the column overflows, we will try
to increase the decimal precision by 10. But if it's reaching 38 (the max precision), the
overflow still could happen. Right now, if such case happens, the following exception will
throw since hive is writing incorrect data.

{noformat}
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
        at org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.readVInt(LazyBinaryUtils.java:314)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.checkObjectByteInfo(LazyBinaryUtils.java:219)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.parse(LazyBinaryStruct.java:142)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
{noformat}

  was:
When we handle sum over decimal datatype in a groupby statement, it will possibly yield overflow
if we use original data type. 

We need to change precision/scale for sum() result or give an error/warning if such change
is not possible.


> Handle the overflow case for decimal datatype for sum()
> -------------------------------------------------------
>
>                 Key: HIVE-13423
>                 URL: https://issues.apache.org/jira/browse/HIVE-13423
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 2.0.0
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>
> When a column col1 defined as decimal and if the sum of the column overflows, we will
try to increase the decimal precision by 10. But if it's reaching 38 (the max precision),
the overflow still could happen. Right now, if such case happens, the following exception
will throw since hive is writing incorrect data.
> {noformat}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
>         at org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.readVInt(LazyBinaryUtils.java:314)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.checkObjectByteInfo(LazyBinaryUtils.java:219)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.parse(LazyBinaryStruct.java:142)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message