hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HIVE-72) wrong results if partition pruning not strict and no mep-reduce job needed
Date Thu, 20 Nov 2008 22:30:44 GMT

     [ https://issues.apache.org/jira/browse/HIVE-72?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

dhruba borthakur updated HIVE-72:

       Resolution: Fixed
    Fix Version/s: 0.20.0
     Hadoop Flags: [Reviewed]
           Status: Resolved  (was: Patch Available)

I just committed this. Thanks Namit!

> wrong results if partition pruning not strict and no mep-reduce job needed
> --------------------------------------------------------------------------
>                 Key: HIVE-72
>                 URL: https://issues.apache.org/jira/browse/HIVE-72
>             Project: Hadoop Hive
>          Issue Type: Bug
>            Reporter: Namit Jain
>            Assignee: Namit Jain
>             Fix For: 0.20.0
>         Attachments: patch1.mapred.txt
> Suppose T is a partitioned table on ds, where ds is a string column, the following queries:
>  SELECT a.* FROM T a WHERE a.ds=2008-09-08 LIMIT 1;
>  SELECT a.* FROM T a WHERE a.ds=2008-11-10 LIMIT 1;
> return the first row from the first partition.
> This is because of the typecast to double.
> for a.ds=2008-01-01 or anything (a.ds=1),
>  evaluate (Double, Double) is invoked at partition pruning.
> Since '2008-11-01' is not a valid double, it is converted to a null, and therefore the
result of pruning returns null (unknown) - not FALSE.
> All unknowns are also accepted, therefore all partitions are accepted which explains
this behavior.
> filter is not invoked since it is a select * query, so map-reduce job is started.
> We just turn off this optimization if pruning indicates that there can be unknown partitions.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message