hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <>
Subject [jira] [Created] (HIVE-9097) Support runtime skew join for more queries [Spark Branch]
Date Mon, 15 Dec 2014 01:55:13 GMT
Rui Li created HIVE-9097:

             Summary: Support runtime skew join for more queries [Spark Branch]
                 Key: HIVE-9097
             Project: Hive
          Issue Type: Bug
            Reporter: Rui Li

After HIVE-8913, runtime skew join is enabled for spark. But currently the optimization only
supports the simplest case where join is the leave ReduceWork in a work graph. This is because
the results from the original join and the conditional map join have to be unioned to feed
to downstream works, which can be a little tricky for spark.
This JIRA is to research and find a way to relax the above restriction. A possible solution
is to break the original task into two tasks on the join work, and insert the conditional
task in between.

This message was sent by Atlassian JIRA

View raw message