impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Impala Public Jenkins (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-5644,IMPALA-5810: Min reservation improvements
Date Tue, 22 Aug 2017 08:27:12 GMT
Impala Public Jenkins has submitted this change and it was merged.

Change subject: IMPALA-5644,IMPALA-5810: Min reservation improvements

IMPALA-5644,IMPALA-5810: Min reservation improvements

Rejects queries during admission control if:
* the largest (across all backends) min buffer reservation is
  greater than the query mem_limit or buffer_pool_limit
* the sum of the min buffer reservations across the cluster
  is larger than the pool max mem resources

There are some other interesting cases to consider later:
* every per-backend min buffer reservation is less than the
  associated backend's process mem_limit; the current
  admission control code doesn't know about other backend's
  proc mem_limits.

Also reduces minimum non-reservation memory (IMPALA-5810).
See the JIRA for experimental results that show this
slightly improves min memory requirements for small queries.
One reason to tweak this is to compensate for the fact that
BufferedBlockMgr didn't count small buffers against the
BlockMgr limit, but BufferPool counts all buffers against

* Adds new test cases in
* Adds BE tests in reservation-tracker-test for the
  reservation-util code.

Change-Id: Iabe87ce8f460356cfe4d1be4d7092c5900f9d79b
Reviewed-by: Matthew Jacobs <>
Tested-by: Impala Public Jenkins
M be/src/runtime/bufferpool/CMakeLists.txt
M be/src/runtime/bufferpool/
A be/src/runtime/bufferpool/
A be/src/runtime/bufferpool/reservation-util.h
M be/src/runtime/
M be/src/scheduling/
M be/src/scheduling/admission-controller.h
M be/src/scheduling/scheduler.h
M common/thrift/
M testdata/workloads/functional-query/queries/QueryTest/analytic-fns.test
M testdata/workloads/functional-query/queries/QueryTest/spilling-aggs.test
M tests/custom_cluster/
M tests/query_test/
13 files changed, 298 insertions(+), 104 deletions(-)

  Impala Public Jenkins: Verified
  Matthew Jacobs: Looks good to me, approved

To view, visit
To unsubscribe, visit

Gerrit-MessageType: merged
Gerrit-Change-Id: Iabe87ce8f460356cfe4d1be4d7092c5900f9d79b
Gerrit-PatchSet: 11
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Matthew Jacobs <>
Gerrit-Reviewer: Dan Hecht <>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Matthew Jacobs <>
Gerrit-Reviewer: Tim Armstrong <>

View raw message