impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-2.6.0_5.8.0) IMPALA-3242: Remove most usages of RuntimeState::SetMemLimitExceeded()
Date Thu, 19 May 2016 18:36:35 GMT
Michael Ho has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/3140

Change subject: IMPALA-3242: Remove most usages of RuntimeState::SetMemLimitExceeded()
......................................................................

IMPALA-3242: Remove most usages of RuntimeState::SetMemLimitExceeded()

There are multiple places in the code which call
RuntimeState::SetMemLimitExceeded(). Most of them are
unnecessary as the error status constructed will eventually
be propagated up the tree of exec nodes. There is no obvious
reason to treat query memory limit exceeded differently.
In some cases such as scan-node, calling SetMemLimitExceeded()
is actually confusing as all scanner threads may pick up error
status when any thread exceeds query memory limit, causing a
lot of noise in the log.

This change replaces most calls to RuntimeState::SetMemLimitExceeded()
with MemTracker::MemLimitExceeded(). The remaining places are:
the old hash table code, the UDF framework and QueryMaintenance()
which checks for memory limit periodically. The query maintenance
case will be removed eventually once IMPALA-2399 is fixed.

Change-Id: Ic0ca128c768d1e73713866e8c513a1b75e6b4b59
---
M be/src/exec/hash-table.cc
M be/src/exec/hash-table.h
M be/src/exec/hdfs-rcfile-scanner.cc
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/partitioned-aggregation-node.cc
M be/src/exec/partitioned-aggregation-node.h
M be/src/exec/partitioned-hash-join-node.cc
M be/src/runtime/plan-fragment-executor.cc
M be/src/runtime/runtime-state.h
M be/src/service/query-exec-state.cc
10 files changed, 49 insertions(+), 54 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/40/3140/1
-- 
To view, visit http://gerrit.cloudera.org:8080/3140
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic0ca128c768d1e73713866e8c513a1b75e6b4b59
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Michael Ho <kwho@cloudera.com>

Mime
View raw message