ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nisanth Simon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMBARI-21124) Used Average Memory in Ambari dashborad is different from Used Average got from free command in RHEL 7
Date Thu, 25 May 2017 05:09:04 GMT
Nisanth Simon created AMBARI-21124:
--------------------------------------

             Summary: Used Average Memory in Ambari dashborad is different from Used Average
got from free command in RHEL 7
                 Key: AMBARI-21124
                 URL: https://issues.apache.org/jira/browse/AMBARI-21124
             Project: Ambari
          Issue Type: Bug
          Components: ambari-metrics
    Affects Versions: 2.2.0
         Environment: RHEL 7
            Reporter: Nisanth Simon
            Priority: Minor


There is a change in formula for calculating memory_used in "free command" for RHEL 6 &
RHEL 7. 

In RHEL 6, memory_used = ( total - free). You can use "man free" on RHEL 6, to get this formula.

[root@host1 ~]# free               
             total              used        free                shared    buffers     cached
Mem:      24607596            1799392   22808204                 6368      95624     608592
-/+ buffers/cache:    1095176   23512420
Swap:      4194300          0    4194300
[root@host1 ~]#

In the above output, [used =  total -free = 24607596 - 22808204  = 1799392]

>From RHEL 7 onwards, there is an extra attribute (MemAvailable) in /proc/meminfo. In RHEL
7, the memory_used = (total - free - buffers - cache). Refer the link for more details - https://access.redhat.com/solutions/406773
for formula.


[root@host2 ~]# free           
              total               used        free                shared  buff/cache   available
Mem:       24524432     1002640    22724020       28232      797772    23247360
Swap:       4194300           0     4194300
[root@host2 ~]#

In the above output,  [used =  total - free - buffers - cache = 24524432 - 22724020 - 797772
= 22724020]


The Ambari Code calculate the memory used using the formula (mem_used = psutil.virtual_memory().used)
. Refer the code, /usr/lib/python2.6/site-packages/resource_monitoring/core/host_info.py


 def get_mem_info(self):
    """
    Return memory statistics at current time
    """

    mem_stats = psutil.virtual_memory()
    swap_stats = psutil.swap_memory()
    disk_usage = self.get_combined_disk_usage()
    mem_total = self.__host_static_info.get('mem_total')

    bytes2kilobytes = lambda x: x / 1024

    return {
      'mem_total': bytes2kilobytes(mem_total) if mem_total else 0,
      'mem_used': bytes2kilobytes(mem_stats.used) if hasattr(mem_stats, 'used') else 0,
      'mem_free': bytes2kilobytes(mem_stats.free) if hasattr(mem_stats, 'free') else 0,
      'mem_shared': bytes2kilobytes(mem_stats.shared) if hasattr(mem_stats, 'shared') else
0,
      'mem_buffered': bytes2kilobytes(mem_stats.buffers) if hasattr(mem_stats, 'buffers')
else 0,
      'mem_cached': bytes2kilobytes(mem_stats.cached) if hasattr(mem_stats, 'cached') else
0,
      'swap_free': bytes2kilobytes(swap_stats.free) if hasattr(swap_stats, 'free') else 0,
      'disk_free' : disk_usage.get("disk_free"),
      # todo: cannot send string
      #'part_max_used' : disk_usage.get("max_part_used")[0],
      'disk_total' : disk_usage.get("disk_total")
    }
  pass

The Ambari Code psutil.virtual_memory() calculates the memory_used using the formula (total
-free). In the code, /usr/lib/python2.6/site-packages/resource_monitoring/psutil/psutil/_pslinux.py

def virtual_memory():
    total, free, buffers, shared, _, _ = cext.linux_sysinfo()
    cached = active = inactive = None
    f = open('/proc/meminfo', 'rb')
    CACHED, ACTIVE, INACTIVE = b("Cached:"), b("Active:"), b("Inactive:")
    try:
        for line in f:
            if line.startswith(CACHED):
                cached = int(line.split()[1]) * 1024
            elif line.startswith(ACTIVE):
                active = int(line.split()[1]) * 1024
            elif line.startswith(INACTIVE):
                inactive = int(line.split()[1]) * 1024
            if (cached is not None
                    and active is not None
                    and inactive is not None):
                break
        else:
            # we might get here when dealing with exotic Linux flavors, see:
            # http://code.google.com/p/psutil/issues/detail?id=313
            msg = "'cached', 'active' and 'inactive' memory stats couldn't " \
                  "be determined and were set to 0"
            warnings.warn(msg, RuntimeWarning)
            cached = active = inactive = 0
    finally:
        f.close()
    avail = free + buffers + cached
    used = total - free
    percent = usage_percent((total - avail), total, _round=1)
    return svmem(total, avail, percent, used, free,
                 active, inactive, buffers, cached)


>From the above, we can conclude that the memory_used_average displayed in Ambari dashboard
will be same with free command average for the cluster for RHEL 6.

In RHEL 7, the memory_used_average displayed in Ambari dashboard will be different from free
command average for the cluster.


Ambari adheres to the formula used by free command of RHEL 6 not of RHEL 7.





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message