impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-3208: max row size option
Date Mon, 21 Aug 2017 22:30:14 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-3208: max_row_size option

Patch Set 11:

Commit Message:

PS11, Line 18: The default value is 512KB. I picked that number because it doesn't
             : increase minimum reservations *too* much but should be large enough
             : for almost all reasonable workloads.
> what's the advantage of picking that below the default buffer size? Oh, I g
Yeah exactly, it makes a big difference for small buffers (e.g. 64kb).
File be/src/exec/partitioned-aggregation-node.h:

Line 741:           resource_profile_.max_row_buffer_size * 2;
> if we're going to use the resource profile to compute this, any reason not 
Yeah these computations are redundant with the planner. I could just rip them out but in some
ways it seemed nice to have this code closer to the implementation as a way to document and
enforce that the planner's calculation matches the backend's logic.
File be/src/service/

PS11, Line 522: Spillable
> should that say "Min spillable"?
File common/thrift/ImpalaInternalService.thrift:

Line 270:   60: optional i64 max_row_size = 524288;
> doesn't have to be in this commit, but it'd be nice to document the interac
I expanded the comment here slightly. I think there's a distinction between the meaning of
the option, which we should document clearly here, and the way it is implemented for different
operators, which should be documented somewhere else.
File fe/src/main/java/org/apache/impala/planner/

PS11, Line 323: max-size buffers
> let's spell that out a bit more:
File fe/src/main/java/org/apache/impala/planner/

PS11, Line 235: max-size buffers
> same

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: Ic70f6dddbcef124bb4b329ffa2e42a74a1826570
Gerrit-PatchSet: 11
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <>
Gerrit-Reviewer: Dan Hecht <>
Gerrit-Reviewer: Matthew Jacobs <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-HasComments: Yes

View raw message