impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Impala Public Jenkins (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5158: Part 1: include untracked memory in MemTracker dumps
Date Wed, 10 May 2017 00:05:49 GMT
Impala Public Jenkins has submitted this change and it was merged.

Change subject: IMPALA-5158: Part 1: include untracked memory in MemTracker dumps
......................................................................


IMPALA-5158: Part 1: include untracked memory in MemTracker dumps

The MemTracker dumps are easier to understand if all children of the
process tracker actually add up to the process consumption. There is
always a discrepancy, which corresponds (mostly) to untracked memory.
This commit adds a new line item in the MemTracker dump which is the
process consumption value minus the sum of all its children.

Here is an example of the new output:

  Process: Limit=8.35 GB Total=123.74 MB Peak=219.93 MB
    Free Disk IO Buffers: Total=1.37 MB Peak=1.37 MB
    RequestPool=default-pool: Total=609.09 KB Peak=10.14 MB
      Query(fd40f6b7542f7e7c:4aef272300000000): Total=609.09 KB Peak=1.19 MB
	Fragment fd40f6b7542f7e7c:4aef272300000000: Total=609.09 KB Peak=1.19 MB
	  AGGREGATION_NODE (id=3): Total=24.00 KB Peak=36.00 KB
	    Exprs: Total=4.00 KB Peak=4.00 KB
	  NESTED_LOOP_JOIN_NODE (id=2): Total=412.90 KB Peak=412.90 KB
	    Nested Loop Join Builder: Total=392.00 KB Peak=392.00 KB
	  HDFS_SCAN_NODE (id=1): Total=160.00 KB Peak=768.00 KB
	    Exprs: Total=4.00 KB Peak=4.00 KB
	  HDFS_SCAN_NODE (id=0): Total=0 Peak=344.00 KB
	  PLAN_ROOT_SINK: Total=0 Peak=0
	  CodeGen: Total=4.19 KB Peak=267.50 KB
	Block Manager: Limit=6.68 GB Total=0 Peak=0
    RequestPool=fe-eval-exprs: Total=0 Peak=4.00 KB
    Untracked Memory: Total=121.78 MB

Change-Id: I77e5b2ef5f85f82ee0226e4dad638233fec9428f
Reviewed-on: http://gerrit.cloudera.org:8080/6820
Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
Tested-by: Impala Public Jenkins
---
M be/src/runtime/mem-tracker.cc
M be/src/runtime/mem-tracker.h
2 files changed, 46 insertions(+), 24 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Tim Armstrong: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/6820
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I77e5b2ef5f85f82ee0226e4dad638233fec9428f
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Mostafa Mokhtar <mmokhtar@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message