hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-14324) ORC PPD for floats is broken
Date Mon, 25 Jul 2016 08:32:20 GMT

     [ https://issues.apache.org/jira/browse/HIVE-14324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prasanth Jayachandran updated HIVE-14324:
-----------------------------------------
    Target Version/s: 2.1.0, 2.2.0  (was: 1.3.0, 2.1.0, 2.2.0)

> ORC PPD for floats is broken
> ----------------------------
>
>                 Key: HIVE-14324
>                 URL: https://issues.apache.org/jira/browse/HIVE-14324
>             Project: Hive
>          Issue Type: Bug
>          Components: ORC
>    Affects Versions: 1.3.0, 2.0.0, 2.1.0, 2.2.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Critical
>
> ORC stores min/max stats, bloom filters by passing floats as doubles using java's widening
conversion. So if we write a float value of 0.22 to ORC file, the min/max stats and bloom
filter will use 0.2199999988079071 double value.
> But when we do PPD, SARG creates literals by converting float to string and then to double
which compares 0.22 to 0.2199999988079071 and fails PPD evaluation. 
> {code}
> hive> create table orc_float (f float) stored as orc;
> hive> insert into table orc_float values(0.22);
> hive> set hive.optimize.index.filter=true;
> hive> select * from orc_float where f=0.22;
> OK
> hive> set hive.optimize.index.filter=false;
> hive> select * from orc_float where f=0.22;
> OK
> 0.22
> {code}
> This is not a problem for doubles and decimals.
> This issue was introduced in HIVE-8460 but back then there was no strict type check when
SARGs are created and also PPD evaluation does not convert to column type. But now predicate
leaf creation in SARG enforces strict type check for boxed literals and predicate type and
PPD evaluation converts stats and constants to column type (predicate).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message