hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-4768) getAvailablePhysicalMemorySize can be inaccurate on linux
Date Fri, 04 Mar 2016 19:56:40 GMT
Nathan Roberts created YARN-4768:

             Summary: getAvailablePhysicalMemorySize can be inaccurate on linux
                 Key: YARN-4768
                 URL: https://issues.apache.org/jira/browse/YARN-4768
             Project: Hadoop YARN
          Issue Type: Bug
          Components: nodemanager
    Affects Versions: 2.7.2, 3.0.0
         Environment: Linux
            Reporter: Nathan Roberts
            Assignee: Nathan Roberts

Algorithm currently uses "MemFree" + "Inactive" from /proc/meminfo

"Inactive" may not be a very good indication of how much memory can be readily freed because
it contains both:
- Pages mapped with MAP_SHARED|MAP_ANONYMOUS (regardless of whether they're being actively
accessed or not. Unclear to me why this is the case...)
- Pages mapped MAP_PRIVATE|MAP_ANONYMOUS that have not been accessed recently

Both of these types of pages probably shouldn't be considered "Available".

"Inactive(file)" would seem more accurate but it's not available in all kernel versions. To
keep things simple, maybe just use "Inactive(file)" if available, otherwise fallback to "Inactive".

This message was sent by Atlassian JIRA

View raw message