hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Pivovarov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10711) Tez HashTableLoader attempts to allocate more memory than available when HIVECONVERTJOINNOCONDITIONALTASKTHRESHOLD exceeds process max mem
Date Fri, 22 May 2015 05:13:17 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555585#comment-14555585
] 

Alexander Pivovarov commented on HIVE-10711:
--------------------------------------------

1. should we put the following code inside "if" block where hashtableMemoryUsage is actually
used
{code}
+    float hashtableMemoryUsage = HiveConf.getFloatVar(
+        hconf, HiveConf.ConfVars.HIVEHASHTABLEFOLLOWBYGBYMAXMEMORYUSAGE);
{code}


> Tez HashTableLoader attempts to allocate more memory than available when HIVECONVERTJOINNOCONDITIONALTASKTHRESHOLD
exceeds process max mem
> ------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-10711
>                 URL: https://issues.apache.org/jira/browse/HIVE-10711
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Jason Dere
>            Assignee: Mostafa Mokhtar
>             Fix For: 1.2.1
>
>         Attachments: HIVE-10711.1.patch, HIVE-10711.2.patch, HIVE-10711.3.patch
>
>
> Tez HashTableLoader bases its memory allocation on HIVECONVERTJOINNOCONDITIONALTASKTHRESHOLD.
If this value is largeer than the process max memory then this can result in the HashTableLoader
trying to use more memory than available to the process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message