hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sahil Takiar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-17958) spark_dynamic_partition_pruning.q fails when hive.tez.dynamic.semijoin.reduction is false
Date Wed, 01 Nov 2017 19:59:02 GMT
Sahil Takiar created HIVE-17958:
-----------------------------------

             Summary: spark_dynamic_partition_pruning.q fails when hive.tez.dynamic.semijoin.reduction
is false
                 Key: HIVE-17958
                 URL: https://issues.apache.org/jira/browse/HIVE-17958
             Project: Hive
          Issue Type: Sub-task
          Components: Spark
            Reporter: Sahil Takiar
            Assignee: Sahil Takiar


Looks like {{RedundantDynamicPruningConditionsRemoval}} causes DPP to be disabled in a few
cases (not sure why). When {{hive.tez.dynamic.semijoin.reduction}} is {{true}} (the default),
then this rule is disabled so the normal tests don't hit this issue.

But when I disable {{hive.tez.dynamic.semijoin.reduction}} then the following query no longer
fully triggers DPP:

{code}
EXPLAIN select count(*) from srcpart join srcpart_date on (srcpart.ds = srcpart_date.ds) join
srcpart_hour on (srcpart.hr = srcpart_hour.hr)
5777 where srcpart_date.`date` = '2008-04-08' and srcpart_hour.hour = 11 and srcpart.hr =
11
{code}

There should be two DPP sinks, but when the config is set to false, there is only one.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message