hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-15239) hive on spark combine equivalentwork get wrong result because of tablescan operation compare
Date Wed, 23 Nov 2016 15:30:59 GMT

     [ https://issues.apache.org/jira/browse/HIVE-15239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rui Li updated HIVE-15239:
--------------------------
    Attachment: HIVE-15239.1.patch

Thanks [~wenli] for reporting the issue.
The problem is we can really tell whether two TS are equivalent if they have same schema,
alias, etc. So the patch checks the path to partition info of MapWorks before checking operators.

> hive on spark combine equivalentwork get wrong result because of  tablescan operation
compare
> ---------------------------------------------------------------------------------------------
>
>                 Key: HIVE-15239
>                 URL: https://issues.apache.org/jira/browse/HIVE-15239
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>    Affects Versions: 1.2.0, 2.1.0
>            Reporter: wangwenli
>            Assignee: Rui Li
>         Attachments: HIVE-15239.1.patch
>
>
> env: hive on spark engine
> reproduce step:
> {code}
> create table a1(KEHHAO string, START_DT string) partitioned by (END_DT string);
> create table a2(KEHHAO string, START_DT string) partitioned by (END_DT string);
> alter table a1 add partition(END_DT='20161020');
> alter table a1 add partition(END_DT='20161021');
> insert into table a1 partition(END_DT='20161020') values('2000721360','20161001');
> SELECT T1.KEHHAO,COUNT(1) FROM ( 
> SELECT KEHHAO FROM a1 T 
> WHERE T.KEHHAO = '2000721360' AND '20161018' BETWEEN T.START_DT AND T.END_DT-1 
> UNION ALL 
> SELECT KEHHAO FROM a2 T
> WHERE T.KEHHAO = '2000721360' AND '20161018' BETWEEN T.START_DT AND T.END_DT-1 
> ) T1 
> GROUP BY T1.KEHHAO 
> HAVING COUNT(1)>1; 
> +-------------+------+--+
> |  t1.kehhao  | _c1  |
> +-------------+------+--+
> | 2000721360  | 2    |
> +-------------+------+--+
> {code}
> the result should be none record



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message