hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3403) user should not specify mapjoin to perform sort-merge bucketed join
Date Sun, 10 Feb 2013 06:33:13 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13575358#comment-13575358
] 

Namit Jain commented on HIVE-3403:
----------------------------------

Yes, but we need to get there slowly - the idea is to get rid of the map-join hint eventually,
and this jira is a step in that direction.
c) above will be achieved with this jira. 

With HIVE-3891 (which is also patch available), a backup task will be created for the map-join,
just like a) above.

I agree, I am adding more config. variables right now - the idea behind them is to test them
slowly in deployment.
In case there are bugs, it would be easy to turn them off. Although there are more configs,
it is still much easier than
hints - once these configs are stabilized, we can change the defaults of these configs, and
we dont need to specify anything
(hint) per query.

Doing b) automatically requires most work, among a), b) and c), since we need to load all
the buckets - theoritically, we can
load the largest bucket and decide based on that. That is definitely a big enough patch, and
should not be part of this jira.
                
> user should not specify mapjoin to perform sort-merge bucketed join
> -------------------------------------------------------------------
>
>                 Key: HIVE-3403
>                 URL: https://issues.apache.org/jira/browse/HIVE-3403
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Namit Jain
>            Assignee: Namit Jain
>         Attachments: hive.3403.10.patch, hive.3403.11.patch, hive.3403.12.patch, hive.3403.13.patch,
hive.3403.14.patch, hive.3403.15.patch, hive.3403.16.patch, hive.3403.17.patch, hive.3403.18.patch,
hive.3403.19.patch, hive.3403.1.patch, hive.3403.21.patch, hive.3403.22.patch, hive.3403.23.patch,
hive.3403.24.patch, hive.3403.25.patch, hive.3403.26.patch, hive.3403.2.patch, hive.3403.3.patch,
hive.3403.4.patch, hive.3403.5.patch, hive.3403.6.patch, hive.3403.7.patch, hive.3403.8.patch,
hive.3403.9.patch
>
>
> Currently, in order to perform a sort merge bucketed join, the user needs
> to set hive.optimize.bucketmapjoin.sortedmerge to true, and also specify the 
> mapjoin hint.
> The user should not specify any hints.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message