hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "liyunzhang_intel (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-16862) Implement a similar feature like "hive.tez.dynamic.semijoin.reduction" in hive on spark
Date Thu, 08 Jun 2017 22:13:18 GMT
liyunzhang_intel created HIVE-16862:
---------------------------------------

             Summary: Implement a similar feature like "hive.tez.dynamic.semijoin.reduction"
in hive on spark
                 Key: HIVE-16862
                 URL: https://issues.apache.org/jira/browse/HIVE-16862
             Project: Hive
          Issue Type: Bug
            Reporter: liyunzhang_intel


Currently if we enable "hive.tez.dynamic.semijoin.reduction" (the default value is true) in
hive on spark, following script fail
{code}
set hive.optimize.ppd=true;
set hive.ppd.remove.duplicatefilters=true;
set hive.spark.dynamic.partition.pruning=true;
set hive.optimize.metadataonly=false;
set hive.optimize.index.filter=true;
set hive.strict.checks.cartesian.product=false;
set hive.spark.dynamic.partition.pruning=true;

-- multiple sources, single key
select count(*) from srcpart join srcpart_date on (srcpart.ds = srcpart_date.ds) join srcpart_hour
on (srcpart.hr = srcpart_hour.hr) {code}
{code}
the reason why this fail see HIVE-16780, currently we only disable "hive.tez.dynamic.semijoin.reduction"
when running hive on spark to pass the test.  Later we can implement a similar feature like
what hive on tez does.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message