hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao (JIRA)" <>
Subject [jira] [Created] (HIVE-8233) multi-table insertion doesn't work with ForwardOperator [Spark Branch]
Date Tue, 23 Sep 2014 17:46:35 GMT
Chao created HIVE-8233:

             Summary: multi-table insertion doesn't work with ForwardOperator [Spark Branch]
                 Key: HIVE-8233
             Project: Hive
          Issue Type: Bug
          Components: Spark
            Reporter: Chao

Right now, for multi-table insertion, we will start from multiple FileSinkOperators, and break
from their lowest common ancestor, adding temporary FileSinkOperator and TableScanOperators.
A special case is when the LCA is a ForwardOperator, in which case we don't break it, since
it's already been optimized.

However, there's a issue, considering the following plan:

     /   \
   RS_1  RS_2
    |     |
   ...   ...
    |     |
   FS_1  FS_2

In this case, {{FOR}} is the LCA, and the plan will still be a single one. However, {{RS_0}}
leads to both {{RS_1}} and {{RS_2}}. Because of the issue in HIVE-7731 and HIVE-8118, both
downstream branches will get duplicated (and same) results.

This message was sent by Atlassian JIRA

View raw message