drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vitalii Diravka (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (DRILL-6094) Decimal data type enhancements
Date Fri, 13 Apr 2018 06:17:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-6094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16435914#comment-16435914
] 

Vitalii Diravka edited comment on DRILL-6094 at 4/13/18 6:16 AM:
-----------------------------------------------------------------

Looks like ARROW-61 is not an issue for Drill anymore.
 Drill 1.6.0 version:
{code:java}
SELECT CAST('922337203685477587' as DECIMAL(18, 0)) > CAST('0.1' as DECIMAL(18, 1)) IS_FIRST_GREATER
FROM (VALUES(1))
--------------------------------------
| IS_FIRST_GREATER<BIT(REQUIRED)>    |
--------------------------------------
| false                              |
--------------------------------------

select _DECIMAL_decimal18 > _DECIMAL_decimal18_2 IS_FIRST_GREATER, _DECIMAL_decimal18,
_DECIMAL_decimal18_2 from dfs.`/tmp/parquet/drill/parquet_test_file_simple` t
----------------------------------------------------------------------------------------------------------------
| IS_FIRST_GREATER<BIT(REQUIRED)>        | _DECIMAL_decimal18<DECIMAL18(REQUI | _DECIMAL_decimal18_2<DECIMAL18(REQ
|
----------------------------------------------------------------------------------------------------------------
| false                              | 922337203685477587                 | 0.1          
                     |
----------------------------------------------------------------------------------------------------------------
{code}
Current Drill master:
{code:java}
SELECT CAST('922337203685477587' as DECIMAL(18, 0)) > CAST('0.1' as DECIMAL(18, 1)) IS_FIRST_GREATER
FROM (VALUES(1))
----------------------------------------------------------------------------------------------------------------
IS_FIRST_GREATER<BIT(REQUIRED)>
true

select _DECIMAL_decimal18 > _DECIMAL_decimal18_2 IS_FIRST_GREATER, _DECIMAL_decimal18,
_DECIMAL_decimal18_2" +
          " from dfs.root.`/tmp/parquet/drill/parquet_test_file_simple` t
----------------------------------------------------------------------------------------------------------------
IS_FIRST_GREATER<BIT(REQUIRED)>,_DECIMAL_decimal18<DECIMAL18(REQUIRED)>,_DECIMAL_decimal18_2<DECIMAL18(REQUIRED)>
true,922337203685477587,0.1
{code}
For the first query Calcite simplifies the predicaste in the project to true and for the second
query casting to FLOAT8 is used now (and FLOAT8 values are compared).

Therefore _adjustScaleMultiply_ is still buggy and isn't needed anymore (after moving to VARDECIMAL)
I would recommend to remove it from the project or to make it deprecated.


was (Author: vitalii):
Looks like ARROW-61 is not an issue for Drill anymore.
 Drill 1.6.0 version:
{code:java}
SELECT CAST('922337203685477587' as DECIMAL(18, 0)) > CAST('0.1' as DECIMAL(18, 1)) IS_FIRST_GREATER
FROM (VALUES(1))
--------------------------------------
| IS_FIRST_GREATER<BIT(REQUIRED)>    |
--------------------------------------
| false                              |
--------------------------------------

select _DECIMAL_decimal18 > _DECIMAL_decimal18_2 IS_FIRST_GREATER, _DECIMAL_decimal18,
_DECIMAL_decimal18_2 from dfs.`/tmp/parquet/drill/parquet_test_file_simple` t
----------------------------------------------------------------------------------------------------------------
| IS_FIRST_GREATER<BIT(REQUIRED)>        | _DECIMAL_decimal18<DECIMAL18(REQUI | _DECIMAL_decimal18_2<DECIMAL18(REQ
|
----------------------------------------------------------------------------------------------------------------
| false                              | 922337203685477587                 | 0.1          
                     |
----------------------------------------------------------------------------------------------------------------
{code}
Current Drill master:
{code:java}
SELECT CAST('922337203685477587' as DECIMAL(18, 0)) > CAST('0.1' as DECIMAL(18, 1)) IS_FIRST_GREATER
FROM (VALUES(1))
----------------------------------------------------------------------------------------------------------------
IS_FIRST_GREATER<BIT(REQUIRED)>
true

select _DECIMAL_decimal18 > _DECIMAL_decimal18_2 IS_FIRST_GREATER, _DECIMAL_decimal18,
_DECIMAL_decimal18_2" +
          " from dfs.root.`/tmp/parquet/drill/parquet_test_file_simple` t
----------------------------------------------------------------------------------------------------------------
IS_FIRST_GREATER<BIT(REQUIRED)>,_DECIMAL_decimal18<DECIMAL18(REQUIRED)>,_DECIMAL_decimal18_2<DECIMAL18(REQUIRED)>
true,922337203685477587,0.1
{code}
For the first query Calcite simplifies the predicaste in the project to true and for the second
query casting to FLOAT8 is used now (and FLOAT8 values are compared).

Therefore _adjustScaleMultiply_ is still buggy and isn't needed anymore (after moving to VARDECIMAL)
I would recommend to remove it from the project.

> Decimal data type enhancements
> ------------------------------
>
>                 Key: DRILL-6094
>                 URL: https://issues.apache.org/jira/browse/DRILL-6094
>             Project: Apache Drill
>          Issue Type: Improvement
>    Affects Versions: 1.12.0
>            Reporter: Volodymyr Vysotskyi
>            Assignee: Volodymyr Vysotskyi
>            Priority: Major
>
> Currently, Decimal types are disabled by default since existing Decimal implementation
has a lot of flaws and performance problems. The goal of this Jira to describe majority of
them and possible ways of improving existing implementation to be able to enable Decimal data
types by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message