hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-8208) Multi-table insertion optimization #1: don't always break operator tree. [Spark Branch]
Date Fri, 24 Oct 2014 17:53:34 GMT

     [ https://issues.apache.org/jira/browse/HIVE-8208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xuefu Zhang resolved HIVE-8208.
-------------------------------
       Resolution: Won't Fix
    Fix Version/s: spark-branch

With HIVE-8118, this is no longer needed.

> Multi-table insertion optimization #1: don't always break operator tree. [Spark Branch]
> ---------------------------------------------------------------------------------------
>
>                 Key: HIVE-8208
>                 URL: https://issues.apache.org/jira/browse/HIVE-8208
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Chao
>             Fix For: spark-branch
>
>
> Currently, with the current patch of multi-table insertion, it will break whenever there
exists one TableScanOperator that can leads to multiple FileSinkOperators. Then, it identifies
the lowest common ancestor (LCA), and breaks the tree there, creating same number of child
SparkTasks as the number of FileSinkOperators.
> However, in the following situation it's better not to break the operator tree:
> Of all the paths from these FileSinkOperators to the LCA, if ReduceSinkOperator only
exist in 0 or 1 path of them.
> In this case, we can do it in one spark job, and no need to break the operator tree.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message