hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Sun (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-11297) Combine op trees for partition info generating tasks [Spark branch]
Date Fri, 17 Jul 2015 20:50:04 GMT
Chao Sun created HIVE-11297:
-------------------------------

             Summary: Combine op trees for partition info generating tasks [Spark branch]
                 Key: HIVE-11297
                 URL: https://issues.apache.org/jira/browse/HIVE-11297
             Project: Hive
          Issue Type: Bug
    Affects Versions: spark-branch
            Reporter: Chao Sun


Currently, for dynamic partition pruning in Spark, if a small table generates partition info
for more than one partition columns, multiple operator trees are created, which all start
from the same table scan op, but have different spark partition pruning sinks.

As an optimization, we can combine these op trees and so don't have to do table scan multiple
times.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message