hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16425) Vectorization: unload old hashtables before reloadHashTable
Date Wed, 12 Apr 2017 14:16:41 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965930#comment-15965930
] 

Matt McCline commented on HIVE-16425:
-------------------------------------

+1 LGTM

> Vectorization: unload old hashtables before reloadHashTable
> -----------------------------------------------------------
>
>                 Key: HIVE-16425
>                 URL: https://issues.apache.org/jira/browse/HIVE-16425
>             Project: Hive
>          Issue Type: Bug
>          Components: Vectorization
>            Reporter: Gopal V
>            Assignee: Gopal V
>         Attachments: HIVE-16425.1.patch
>
>
> {code}
>   @Override
>   protected void reloadHashTable(byte pos, int partitionId)
>           throws IOException, HiveException, SerDeException, ClassNotFoundException {
>     // The super method will reload a hash table partition of one of the small tables.
>     // Currently, for native vector map join it will only be one small table.
>     super.reloadHashTable(pos, partitionId);
>     MapJoinTableContainer smallTable = spilledMapJoinTables[pos];
>     vectorMapJoinHashTable = VectorMapJoinOptimizedCreateHashTable.createHashTable(conf,
>         smallTable);
>     needHashTableSetup = true;
>     LOG.info("Created " + vectorMapJoinHashTable.getClass().getSimpleName() + " from
" + this.getClass().getSimpleName());
>     if (isLogDebugEnabled) {
>       LOG.debug(CLASS_NAME + " reloadHashTable!");
>     }
>   }
> {code}
> The super call causes an OOM because of existing memory usage by vectorMapJoinHashTable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message