hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <>
Subject [jira] [Created] (HIVE-16425) Vectorization: unload old hashtables before reloadHashTable
Date Wed, 12 Apr 2017 11:14:41 GMT
Gopal V created HIVE-16425:

             Summary: Vectorization: unload old hashtables before reloadHashTable
                 Key: HIVE-16425
             Project: Hive
          Issue Type: Bug
          Components: Vectorization
            Reporter: Gopal V


  protected void reloadHashTable(byte pos, int partitionId)
          throws IOException, HiveException, SerDeException, ClassNotFoundException {

    // The super method will reload a hash table partition of one of the small tables.
    // Currently, for native vector map join it will only be one small table.
    super.reloadHashTable(pos, partitionId);

    MapJoinTableContainer smallTable = spilledMapJoinTables[pos];

    vectorMapJoinHashTable = VectorMapJoinOptimizedCreateHashTable.createHashTable(conf,
    needHashTableSetup = true;"Created " + vectorMapJoinHashTable.getClass().getSimpleName() + " from " + this.getClass().getSimpleName());

    if (isLogDebugEnabled) {
      LOG.debug(CLASS_NAME + " reloadHashTable!");

The super call causes an OOM because of existing memory usage by vectorMapJoinHashTable.

This message was sent by Atlassian JIRA

View raw message