impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-5417: make I/O buffer queue fixed-size
Date Thu, 14 Sep 2017 21:50:18 GMT
Hello Michael Ho, Dan Hecht,

I'd like you to reexamine a change.  Please visit

to look at the new patch set (#9).

Change subject: IMPALA-5417: make I/O buffer queue fixed-size

IMPALA-5417: make I/O buffer queue fixed-size

This removes the dynamically-varying queue size behaviour in the I/O
manager. The motivation is to bound resource consumption of scans
and make it possible to reserve memory for I/O buffers upfront.

Does some cleanup/documentation of the locking policy. Fix some cases
in ScanRange::GetNext() where members documented as being protected by
ScanRange::lock_ were accessed without holding it. I think the races
were either benign or prevented by holding DiskIoRequestContext::lock_
in practice.

Ran exhaustive build.

Ran the full set of workloads (TPC-H, TPC-DS, targeted) on a 16 node
cluster. Everything was within normal variance.

Change-Id: If7cc3f7199f5320db00b7face97a96cdadb6f83f
M be/src/runtime/disk-io-mgr-internal.h
M be/src/runtime/
M be/src/runtime/
M be/src/runtime/
M be/src/runtime/
M be/src/runtime/disk-io-mgr.h
6 files changed, 289 insertions(+), 342 deletions(-)

  git pull ssh:// refs/changes/08/7408/9
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If7cc3f7199f5320db00b7face97a96cdadb6f83f
Gerrit-PatchSet: 9
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <>
Gerrit-Reviewer: Dan Hecht <>
Gerrit-Reviewer: Matthew Jacobs <>
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Michael Ho <>
Gerrit-Reviewer: Tim Armstrong <>

View raw message