hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-15503) LLAP: Fix use of Runtime.getRuntime.maxMemory in Hive operators
Date Fri, 23 Dec 2016 09:09:58 GMT

     [ https://issues.apache.org/jira/browse/HIVE-15503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prasanth Jayachandran updated HIVE-15503:
-----------------------------------------
    Attachment: HIVE-15503.2.patch

Added TopNHash as well. 

> LLAP: Fix use of Runtime.getRuntime.maxMemory in Hive operators
> ---------------------------------------------------------------
>
>                 Key: HIVE-15503
>                 URL: https://issues.apache.org/jira/browse/HIVE-15503
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 2.2.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Critical
>         Attachments: HIVE-15503.1.patch, HIVE-15503.2.patch, HIVE-15503.WIP.patch
>
>
> {code}
> ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java:    maxHashTblMemory
= (long) (memoryPercentage * Runtime.getRuntime().maxMemory());
> ql/src/java/org/apache/hadoop/hive/ql/exec/TopNHash.java:    // Total Free Memory = maxMemory()
- Used Memory;
> ql/src/java/org/apache/hadoop/hive/ql/exec/TopNHash.java:    long totalFreeMemory = Runtime.getRuntime().maxMemory()
-
> {code}
> This will not work very well with LLAP because of the memory sharing by executors. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message