asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ASTERIXDB-1556) Prefix-based multi-way Fuzzy-join generates an exception.
Date Wed, 03 Aug 2016 16:40:20 GMT

    [ https://issues.apache.org/jira/browse/ASTERIXDB-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15406181#comment-15406181
] 

Chen Li commented on ASTERIXDB-1556:
------------------------------------

[~wangsaeu]: The discussion results make sense to me.  A few questions: (1) Yourkit reported
80MB memory for each operator, which is very consistent with your calculation "INT_SIZE *
2 * table size (10,485,767)."  What does this 80MB really include then?  Hash Table + Data?
 (2) For the case we don't have enough memory, what do we do to "spill tuples to the disk"?
Can you elaborate?  (3) Seem we have two limit parameters, namely "MAX LIMIT OF FRAME" for
data and "MAX LIMIT OF HASH TABLE FRAME" for the hash table.  Can we simply it by having a
single limit?

We can talk again after you become available in a few days.

> Prefix-based multi-way Fuzzy-join generates an exception.
> ---------------------------------------------------------
>
>                 Key: ASTERIXDB-1556
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1556
>             Project: Apache AsterixDB
>          Issue Type: Bug
>            Reporter: Taewoo Kim
>            Assignee: Taewoo Kim
>         Attachments: 2wayjoin.pdf, 2wayjoin.rtf, 2wayjoinplan.rtf, 3wayjoin.pdf, 3wayjoin.rtf,
3wayjoinplan.rtf
>
>
> When we enable prefix-based fuzzy-join and apply the multi-way fuzzy-join ( > 2),
the system generates an out-of-memory exception. 
> Since a fuzzy-join is created using 30-40 lines of AQL codes and this AQL is translated
into massive number of operators (more than 200 operators in the plan for a 3-way fuzzy join),
it could generate out-of-memory exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message