hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <>
Subject [jira] [Commented] (HIVE-16311) Improve the performance for FastHiveDecimalImpl.fastDivide
Date Thu, 06 Apr 2017 15:54:42 GMT


Xuefu Zhang commented on HIVE-16311:

I agree with [~mmccline] that Hive's methodology in dealing with decimal operations is keeping
intermediate result as precise as possible and possibly scaling it back to fit to the precision/scale
of the final result (precision/scale enforcement). (The final result's precision/scale is
statically determined.) I understand that performance may gain if we shortcut the precision/scale
for the intermediate result, but this would violate the original design principle and cause
differences such as 137174.210013717333 vs 137174.210013717330.

Sorry for not being clear from the beginning (I wasn't reading the change carefully), but
I don't think this is a time to make such a small yet significant change.

Trailing zeroes are added purely for display purpose (HIVE-12063). This is rather a neat approach
that's also an important part of Hive decimal support.

On thing to clarify though, is whether the new decimal is slower than the old decimal in division.
If so, we need to find out why and fix. Otherwise, for performance concern regarding division,
we should optimize it or provide a new implementation, while keeping Hive's design principles.

> Improve the performance for FastHiveDecimalImpl.fastDivide
> ----------------------------------------------------------
>                 Key: HIVE-16311
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 2.2.0
>            Reporter: Colin Ma
>            Assignee: Colin Ma
>             Fix For: 3.0.0
>         Attachments: HIVE-16311.001.patch, HIVE-16311.002.patch, HIVE-16311.003.patch,
HIVE-16311.004.patch, HIVE-16311.005.patch, HIVE-16311.withTrailingZero.patch
> FastHiveDecimalImpl.fastDivide is poor performance when evaluate the expression as 12345.67/123.45
> There are 2 points can be improved:
> 1. Don't always use HiveDecimal.MAX_SCALE as scale when do the BigDecimal.divide.
> 2. Get the precision for BigInteger in a fast way if possible.

This message was sent by Atlassian JIRA

View raw message