impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-5715: (mitigation only) defer destruction of MemTrackers
Date Thu, 03 Aug 2017 17:31:49 GMT
Hello Matthew Jacobs,

I'd like you to reexamine a change.  Please visit

to look at the new patch set (#2).

Change subject: IMPALA-5715: (mitigation only) defer destruction of MemTrackers

IMPALA-5715: (mitigation only) defer destruction of MemTrackers

One potential candidate for the bad MemTracker IMPALA-5715 is one owned
by a join build sink. I haven't found a clear root cause, but we can
reduce the probability of bugs like this by deferring teardown of the

This patch defers destruction of the fragment instance, ExecNode,
DataSink and Codegen MemTrackers until query teardown. Instead
MemTracker::Close() is called at the place where the MemTracker
would have been destroyed to check that all memory is released
and enforce that no more memory is consumed. The entire query
MemTracker subtree is then disconnected in whole from the global
tree in QueryState::ReleaseResources(), instead of the MemTrackers
being incrementally disconnected bottom-up.

In cases where the MemTracker is owned by another object, this
required deferring teardown of the owner until query teardown.
E.g. for LlvmCodeGen I added a Close() method to release resources
and deferred calling the destructor.

We want to make this change anyway - see IMPALA-5587.


Change-Id: I205abb0076d1ffd08cb93c0f1671c8b81e7fba0f
M be/src/codegen/
M be/src/codegen/
M be/src/codegen/llvm-codegen.h
M be/src/exec/
M be/src/exec/
M be/src/exec/
M be/src/exec/
M be/src/exprs/
M be/src/runtime/
M be/src/runtime/bufferpool/
M be/src/runtime/
M be/src/runtime/coordinator.h
M be/src/runtime/
M be/src/runtime/
M be/src/runtime/
M be/src/runtime/mem-tracker.h
M be/src/runtime/
M be/src/runtime/
M be/src/runtime/
M be/src/runtime/runtime-state.h
20 files changed, 114 insertions(+), 79 deletions(-)

  git pull ssh:// refs/changes/92/7492/2
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I205abb0076d1ffd08cb93c0f1671c8b81e7fba0f
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <>
Gerrit-Reviewer: Henry Robinson <>
Gerrit-Reviewer: Matthew Jacobs <>
Gerrit-Reviewer: Michael Ho <>
Gerrit-Reviewer: Tim Armstrong <>

View raw message