impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-2.6.0_5.8.0) IMPALA-3242: Remove most usages of RuntimeState::SetMemLimitExceeded()
Date Fri, 20 May 2016 07:59:32 GMT
Hello Internal Jenkins, Tim Armstrong, Dan Hecht,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/3140

to look at the new patch set (#3).

Change subject: IMPALA-3242: Remove most usages of RuntimeState::SetMemLimitExceeded()
......................................................................

IMPALA-3242: Remove most usages of RuntimeState::SetMemLimitExceeded()

There are multiple places in the code which call
RuntimeState::SetMemLimitExceeded(). Most of them are
unnecessary as the error status constructed will eventually
be propagated up the tree of exec nodes. There is no obvious
reason to treat query memory limit exceeded differently.
In some cases such as scan-node, calling SetMemLimitExceeded()
is actually confusing as all scanner threads may pick up error
status when any thread exceeds query memory limit, causing a
lot of noise in the log.

This change replaces most calls to RuntimeState::SetMemLimitExceeded()
with MemTracker::MemLimitExceeded(). The remaining places are:
the old hash table code, the UDF framework and QueryMaintenance()
which checks for memory limit periodically. The query maintenance
case will be removed eventually once IMPALA-2399 is fixed.

Change-Id: Ic0ca128c768d1e73713866e8c513a1b75e6b4b59
---
M be/src/exec/hash-table.cc
M be/src/exec/hash-table.h
M be/src/exec/hdfs-rcfile-scanner.cc
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/partitioned-aggregation-node.cc
M be/src/exec/partitioned-aggregation-node.h
M be/src/exec/partitioned-hash-join-node.cc
M be/src/runtime/plan-fragment-executor.cc
M be/src/runtime/runtime-state.h
M be/src/service/query-exec-state.cc
M common/thrift/generate_error_codes.py
M testdata/workloads/tpch/queries/tpch-outer-joins.test
12 files changed, 60 insertions(+), 61 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/40/3140/3
-- 
To view, visit http://gerrit.cloudera.org:8080/3140
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic0ca128c768d1e73713866e8c513a1b75e6b4b59
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message