hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wangmeng (JIRA)" <>
Subject [jira] [Commented] (HIVE-964) handle skewed keys for a join in a separate job
Date Tue, 22 Jul 2014 08:25:39 GMT


wangmeng commented on HIVE-964:

if the two join tables  have the same big skew key on one value (for example ,select *  from
 table A join B  on,  both table A  and B  have  a lot of  keys on id=1,  in  this
 case ,map join  will OOM),  how  to fix this  case?  Will  it  rollback  to common  join

> handle skewed keys for a join in a separate job
> -----------------------------------------------
>                 Key: HIVE-964
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: He Yongqiang
>             Fix For: 0.6.0
>         Attachments: hive-964-2009-12-17.txt, hive-964-2009-12-28-2.patch, hive-964-2009-12-29-4.patch,
hive-964-2010-01-08.patch, hive-964-2010-01-13-2.patch, hive-964-2010-01-14-3.patch, hive-964-2010-01-15-4.patch
> The skewed keys can be written to a temporary table or file, and a followup conditional
task can be used to perform the join on those keys.
> As a first step, JDBM can be used for those keys

This message was sent by Atlassian JIRA

View raw message