hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venki Korukanti (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-8099) IN operator for partition column fails when the partition column type is DATE
Date Fri, 07 Nov 2014 01:17:33 GMT

     [ https://issues.apache.org/jira/browse/HIVE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Venki Korukanti updated HIVE-8099:
----------------------------------
    Attachment: HIVE-8099.3.patch

[~ashutoshc] & [~sershe]

Attaching new patch (HIVE-8099.3) which completes the fix in ppr. I ran partition related
.q tests ({{ls -l \*partition\*}} in clientpositive directory). No regressions found. However
the following query doesn't work because IN operator is not implicitly casting string literals
to date.

{code}
SELECT * FROM testTbl WHERE date_prt IN ('2014-08-09', '2014-08-08');
{code}

Casting the string literals to date works fine.

{code}
SELECT * FROM testTbl WHERE date_prt IN (CAST('2014-08-09' AS DATE), CAST('2014-08-08' AS
DATE));
{code}

If we want the first query to work, we need to include the fix in HIVE-8099.2.patch.

Let me know if you want me to create reviewboard link.

> IN operator for partition column fails when the partition column type is DATE
> -----------------------------------------------------------------------------
>
>                 Key: HIVE-8099
>                 URL: https://issues.apache.org/jira/browse/HIVE-8099
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.13.0, 0.13.1
>            Reporter: Venki Korukanti
>            Assignee: Venki Korukanti
>             Fix For: 0.14.0
>
>         Attachments: HIVE-8099-ppr-fix.patch, HIVE-8099.1.patch, HIVE-8099.2.patch, HIVE-8099.3.patch
>
>
> Test table DLL:
> {code}
> CREATE TABLE testTbl(col1 string) PARTITIONED BY (date_prt date);
> {code}
> Following query used to work fine in Hive 0.12 as the constant types are 'string' and
partition column type is considered as 'string' throughout the planning and optimization (including
partition pruning).
> {code}
> SELECT * FROM testTbl WHERE date_prt IN ('2014-08-09', '2014-08-08'); 
> {code}
> In trunk the above query fails with:
> {code}
> Line 1:33 Wrong arguments ''2014-08-08'': The arguments for IN should be the same type!
Types are: {date IN (string, string)}
> {code}
> HIVE-6642 changed the SemanticAnalyzer.java to consider partition type given in table
definition instead of hardcoded 'string' type. (Modified [Hive 0.12 code|https://github.com/apache/hive/blob/branch-0.12/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L7778]).
So changed the query as follows to go past the above error:
> {code}
> SELECT * FROM testTbl WHERE date_prt IN (CAST('2014-08-09' AS DATE), CAST('2014-08-08'
AS DATE)); 
> {code}
> Now query goes past the error in SemanticAnalyzer, but hits the same issue (default 'string'
type for partition columns) in partition pruning optimization. (Realted code [here|https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartExprEvalUtils.java#L110]).
>  
> {code}
> 14/09/14 20:07:20 ERROR ql.Driver: FAILED: SemanticException MetaException(message:The
arguments for IN should be the same type! Types are: {string IN (date, date)})
> {code}
> We need to change partition pruning code to consider the partition column as the type
given in table definition.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message