hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15335) Fast Decimal
Date Tue, 20 Dec 2016 22:06:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765383#comment-15765383
] 

Matt McCline commented on HIVE-15335:
-------------------------------------

The HiveDecimal and HiveDecimalWritable support *all* the V1 public methods and fields.  V2
for both adds a bunch of new methods and a few new fields.  Some V2 methods provide faster
ways to do things than V1.  For example, LazySimpleSerializeWrite uses a V2 scratch byte array
method toBytes to generate the display decimal into and then writes the scratch byte array.
 The V1 logic created a throw away String, called getBytes which allocated another throw away
object, and wrote the bytes.

TestHiveDecimal's primary test philosophy is too create the HiveDecimalV1 and HiveDecimal
objects with the same input (e.g. a String, BigDecimal, etc), do the same operation on both
and compare the V1 and V2 decimals.

HiveDecimalWritableV1 and HiveDecimalWritable support all V1 public methods, too.

One of the major performance improvements was doing operations like the new mutateAdd directly
on the HiveDecimalWritable for sum instead of calling getHiveDecimal(), doing an add which
creates another object, and then setting the HiveDecimalWritable with the result.

> Fast Decimal
> ------------
>
>                 Key: HIVE-15335
>                 URL: https://issues.apache.org/jira/browse/HIVE-15335
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>            Reporter: Matt McCline
>            Assignee: Matt McCline
>            Priority: Critical
>         Attachments: HIVE-15335.01.patch, HIVE-15335.02.patch, HIVE-15335.03.patch, HIVE-15335.04.patch,
HIVE-15335.05.patch, HIVE-15335.06.patch, HIVE-15335.07.patch, HIVE-15335.08.patch, HIVE-15335.09.patch,
HIVE-15335.091.patch, HIVE-15335.092.patch, HIVE-15335.093.patch, HIVE-15335.094.patch, HIVE-15335.095.patch,
HIVE-15335.096.patch, HIVE-15335.097.patch, HIVE-15335.098.patch, HIVE-15335.099.patch
>
>
> Replace HiveDecimal implementation that currently represents the decimal internally as
a BigDecimal with a faster version that does not allocate extra objects
> Replace HiveDecimalWritable implementation with a faster version that has new mutable*
calls (e.g. mutableAdd, mutableEnforcePrecisionScale, etc) and stores the result as a fast
decimal instead of a slow byte array containing a serialized BigInteger.
> Provide faster ways to serialize/deserialize decimals.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message