hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siying Dong (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1721) use bloom filters to improve the performance of joins
Date Tue, 02 Nov 2010 18:44:28 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927537#action_12927537
] 

Siying Dong commented on HIVE-1721:
-----------------------------------

Shouldn't bloom filter even more expensive than normal hash table?
Instead of calculating one hash, we'll have to calculate k of them. And calculating the hash,
is the most expensive part for us.
It probably works if most of the key is not in small table and keys and values of small tables
are saved in disks. I'm not sure it is our use scenario.



> use bloom filters to improve the performance of joins
> -----------------------------------------------------
>
>                 Key: HIVE-1721
>                 URL: https://issues.apache.org/jira/browse/HIVE-1721
>             Project: Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: Siying Dong
>
> In case of map-joins, it is likely that the big table will not find many matching rows
from the small table.
> Currently, we perform a hash-map lookup for every row in the big table, which can be
pretty expensive.
> It might be useful to try out a bloom-filter containing all the elements in the small
table.
> Each element from the big table is first searched in the bloom filter, and only in case
of a positive match,
> the small table hash table is explored.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message