hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3086) Skewed Join Optimization
Date Wed, 27 Jun 2012 04:34:44 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401960#comment-13401960
] 

He Yongqiang commented on HIVE-3086:
------------------------------------

'hint' by user has been proven not very useful. Automatically detecting skewed keys, like
what the current skew join processor is doing now, will make it more powerful and useful.

@Nadeem, can you add more details to the wiki about the differences between the existing one
and the one you are working on. The current one can not process the case where a same join
key is skewed in more than one table. Are you targeting those cases? Also there are some problems
with existing skew join opt, can you also try to fix those as part of your project?
                
> Skewed Join Optimization
> ------------------------
>
>                 Key: HIVE-3086
>                 URL: https://issues.apache.org/jira/browse/HIVE-3086
>             Project: Hive
>          Issue Type: New Feature
>            Reporter: Nadeem Moidu
>            Assignee: Nadeem Moidu
>
> During a join operation, if one of the columns has a skewed key, it can cause that particular
reducer to become the bottleneck. The following feature will address it:
> https://cwiki.apache.org/confluence/display/Hive/Skewed+Join+Optimization

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message