hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liyin Tang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row
Date Mon, 15 Nov 2010 07:06:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931995#action_12931995
] 

Liyin Tang commented on HIVE-1642:
----------------------------------

Thanks for reviewing.

1. I will add these parameters in the config xml file.

2. By default hive.auto.convert.join = false right now, all the existing test cases won't
be affected

3. I am also thinking about putting the backup task into task directly, which is the simplest
way to implement this. My only concern is that it will take more than time de/serializing
the task. 

4. I will remove this the print statement.
5. The same as point 3.
6. I will fix it, some svn synchronization problem.

7. Right now the back up task is generated during the execution time. That's why it is not
easy to work with explain task. But if we put backup task into task directly, we can solve
this problem. Also we should set the backup task during the compile time instead of execution
time. The only cost is the task serialization time.

8. Because we need to reuse the code of MapJoinProcessor, which uses join tree and row resolver
to generate the new map join operator. So each time when generating a new map join operator,
we need a deep copy of join tree and op context. Several classes need to be Serializable.

9. I generated these test cases output by set the hive.auto.convert.join = false first, then
reset the flag as true. So I can compare whether the result is correct or not. 
Since right now, the join result is correct, I can add explain into test case.

10.I will fix the conditional task to make it more generic.


> Convert join queries to map-join based on size of table/row
> -----------------------------------------------------------
>
>                 Key: HIVE-1642
>                 URL: https://issues.apache.org/jira/browse/HIVE-1642
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: Liyin Tang
>             Fix For: 0.7.0
>
>         Attachments: hive_1642_1.patch
>
>
> Based on the number of rows and size of each table, Hive should automatically be able
to convert a join into map-join.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message