impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5658: addtl. process/system-wide memory metrics
Date Tue, 25 Jul 2017 00:25:47 GMT
Hello Henry Robinson,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/7472

to look at the new patch set (#4).

Change subject: IMPALA-5658: addtl. process/system-wide memory metrics
......................................................................

IMPALA-5658: addtl. process/system-wide memory metrics

This is meant to help understand how the process is using memory, to
answer questions like:
* Is the Impala daemon using transparent huge pages?
* What is the system THP configuration?
* What is the RSS of the process?
* What is the virtual memory size of the process?

Most of these questions can be answered via other command line or
monitoring, except for the THP usage by the process, but adding
metrics helps consolidate the information in one place.

This commit adds a memory maintenance thread that periodically scrapes
information from the /proc and /sys filesystems and updates the
metrics.

The interfaces used are:
* /proc/<pid>/smaps, which is a documented interface to get detailed
  information about mapped memory:
  https://www.kernel.org/doc/Documentation/filesystems/proc.txt
* /sys/kernel/mm/ config values for transparent huge pages

Change-Id: I13873e305ba464d11dea0d7244a29ff4f332f1a9
---
M be/src/common/init.cc
M be/src/runtime/exec-env.cc
M be/src/util/default-path-handlers.cc
M be/src/util/mem-info.cc
M be/src/util/mem-info.h
M be/src/util/memory-metrics.cc
M be/src/util/memory-metrics.h
M common/thrift/metrics.json
M www/memz.tmpl
9 files changed, 383 insertions(+), 38 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/7472/4
-- 
To view, visit http://gerrit.cloudera.org:8080/7472
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I13873e305ba464d11dea0d7244a29ff4f332f1a9
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Henry Robinson <henry@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message