hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <>
Subject [jira] [Commented] (HIVE-9097) Support runtime skew join for more queries [Spark Branch]
Date Wed, 17 Dec 2014 04:48:15 GMT


Rui Li commented on HIVE-9097:

Thanks [~xuefuz] for the review.

> Support runtime skew join for more queries [Spark Branch]
> ---------------------------------------------------------
>                 Key: HIVE-9097
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Spark
>    Affects Versions: spark-branch
>            Reporter: Rui Li
>            Assignee: Rui Li
>             Fix For: spark-branch
>         Attachments: HIVE-9097.1-spark.patch
> After HIVE-8913, runtime skew join is enabled for spark. But currently the optimization
only supports the simplest case where join is the leaf ReduceWork in a work graph. This is
because the results from the original join and the conditional map join have to be unioned
to feed to downstream works, which can be a little tricky for spark.
> This JIRA is to research and find a way to relax the above restriction. A possible solution
is to break the original task into two tasks on the join work, and insert the conditional
task in between.

This message was sent by Atlassian JIRA

View raw message