hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Ma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16311) Improve the performance for FastHiveDecimalImpl.fastDivide
Date Wed, 29 Mar 2017 07:43:41 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946670#comment-15946670
] 

Colin Ma commented on HIVE-16311:
---------------------------------

[~mmccline], I check the TestHiveDecimal and original design for division at https://cwiki.apache.org/confluence/download/attachments/27362075/Hive_Decimal_Precision_Scale_Support.pdf,
and I think the implementation of OldHiveDecimal is not the same as original design for division.

In OldHiveDecimal, the result scale is always MAX_SCALE as following code, but in the original
design, the result scale is calculated by the formula *max(6, s1 + p2 + 1)*
{code}
public HiveDecimalV1 divide(HiveDecimalV1 dec) {
    return create(bd.divide(dec.bd, MAX_SCALE, RoundingMode.HALF_UP), true);
  }
{code}
For the test case 1/3 in TestHiveDecimal, the result of OldHiveDecimal is 0.33333333333333333333333333333333333333,
and the result of original design is 0.333333. I think the TestHiveDecimal also should be
updated according to the original design.
I agree you to write a native or fast version of Decimal division. Actually, current patch
only sets the result scalar according to the original design, it only benefits the division
whose result scalar is less than MAX_SCALE. For the division between real big Decimal, eg,
*Decimal(38, 19)/Decimal(38, 19)*, there is a little performance improvement. We need a faster
way to do the division like FastHiveDecimalImpl.fastSingleWordDivision, but there is no conflict
with the current patch. What do you think?
Thanks for your comments. 

> Improve the performance for FastHiveDecimalImpl.fastDivide
> ----------------------------------------------------------
>
>                 Key: HIVE-16311
>                 URL: https://issues.apache.org/jira/browse/HIVE-16311
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 2.2.0
>            Reporter: Colin Ma
>            Assignee: Colin Ma
>             Fix For: 3.0.0
>
>         Attachments: HIVE-16311.001.patch
>
>
> FastHiveDecimalImpl.fastDivide is poor performance when evaluate the expression as 12345.67/123.45
> There are 2 points can be improved:
> 1. Don't always use HiveDecimal.MAX_SCALE as scale when do the BigDecimal.divide.
> 2. Get the precision for BigInteger in a fast way if possible.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message