hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deepak Jaiswal (JIRA)" <j...@apache.org>
Subject [jira] [Work started] (HIVE-16791) Tez engine giving inaccurate results on SMB Map joins while map-join and shuffle join gets correct results
Date Mon, 24 Jul 2017 18:04:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Work on HIVE-16791 started by Deepak Jaiswal.
---------------------------------------------
> Tez engine giving inaccurate results on SMB Map joins while map-join and shuffle join
gets correct results
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-16791
>                 URL: https://issues.apache.org/jira/browse/HIVE-16791
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive, HiveServer2
>            Reporter: Saumil Mayani
>            Assignee: Deepak Jaiswal
>         Attachments: sample-data-query.txt, sample-data.tar.gz-aa, sample-data.tar.gz-ab,
sample-data.tar.gz-ac, sample-data.tar.gz-ad
>
>
> SMB Join gives incorrect results. 
> {code}
> SMB-Join
> set hive.execution.engine=tez;
> set hive.enforce.sortmergebucketmapjoin=false;
> set hive.optimize.bucketmapjoin=true;
> set hive.optimize.bucketmapjoin.sortedmerge=true;
> set hive.auto.convert.sortmerge.join=true;
> set hive.auto.convert.join=true;
> set hive.auto.convert.join.noconditionaltask.size=500000;
> OK
> 2016	1	11999639
> 2016	2	18955110
> 2017	2	22217437
> Time taken: 92.647 seconds, Fetched: 3 row(s)
> {code}
> {code}
> MAP-JOIN
> set hive.execution.engine=tez;
> set hive.enforce.sortmergebucketmapjoin=false;
> set hive.optimize.bucketmapjoin=true;
> set hive.optimize.bucketmapjoin.sortedmerge=true;
> set hive.auto.convert.sortmerge.join=true;
> set hive.auto.convert.join=true;
> set hive.auto.convert.join.noconditionaltask.size=50000000;
> OK
> 2016	1	26586093
> 2016	2	17724062
> 2017	2	8862031
> Time taken: 17.49 seconds, Fetched: 3 row(s)
> {code}
> {code}
> Shuffle Join
> set hive.execution.engine=tez;
> set hive.enforce.sortmergebucketmapjoin=false;
> set hive.optimize.bucketmapjoin=true;
> set hive.optimize.bucketmapjoin.sortedmerge=true;
> set hive.auto.convert.sortmerge.join=false;
> set hive.auto.convert.join=false;
> set hive.auto.convert.join.noconditionaltask.size=50000000;
> OK
> 2016	1	26586093
> 2016	2	17724062
> 2017	2	8862031
> Time taken: 38.575 seconds, Fetched: 3 row(s)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message