hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3996) Correctly enforce the memory limit on the multi-table map-join
Date Mon, 18 Feb 2013 05:33:13 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13580424#comment-13580424
] 

Namit Jain commented on HIVE-3996:
----------------------------------

There is a bug in the patch. For sub-queries, we cannot figure out the size at compile time
- for eg. join29.q.
It should have a backup task.

In the new function, getBigTablePosition, you are loosing the variable convertMapJoin, since
it is a local variable.
                
> Correctly enforce the memory limit on the multi-table map-join
> --------------------------------------------------------------
>
>                 Key: HIVE-3996
>                 URL: https://issues.apache.org/jira/browse/HIVE-3996
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>    Affects Versions: 0.11.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>         Attachments: HIVE-3996_2.patch, HIVE-3996.patch
>
>
> Currently with HIVE-3784, the joins are converted to map-joins based on checks of the
table size against the config variable: hive.auto.convert.join.noconditionaltask.size. 
> However, the current implementation will also merge multiple mapjoin operators into a
single task regardless of whether the sum of the table sizes will exceed the configured value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message