impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-3200: Implement suballocator for splitting buffers
Date Tue, 08 Nov 2016 21:33:07 GMT
Tim Armstrong has uploaded a new patch set (#4).

Change subject: IMPALA-3200: Implement suballocator for splitting buffers

IMPALA-3200: Implement suballocator for splitting buffers

This is useful for situations like hash tables, where we want to
make multiple non-spillable allocations of variable size from buffer
pool memory and not incur the overhead of interacting with the global
buffer pool. The allocator subdivides buffers to service allocations
and uses a buddy allocation algorithm to merge freed allocations into
larger chunks. This helps avoid fragmentation and is quite effective
at reusing memory given the typical doubling allocation patterns of
hash tables in partitioned aggs and joins.

The allocator has fairly robust internal consistency checks via
assertions and unique_ptrs. Includes a unit test that exercises
various allocation patterns.

I tested porting hash tables over to allocate memory using an earlier
version of the suballocator, which worked well (was able to run a
wide range of queries successfully with good performance).

Change-Id: I8bfe0e429f67ad273f7c7d0816703a9e6c3da788
M be/src/bufferpool/CMakeLists.txt
A be/src/bufferpool/
A be/src/bufferpool/
A be/src/bufferpool/suballocator.h
M be/src/common/names.h
5 files changed, 788 insertions(+), 0 deletions(-)

  git pull ssh:// refs/changes/15/4715/4
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8bfe0e429f67ad273f7c7d0816703a9e6c3da788
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <>

View raw message