impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Tauber-Marshall (Code Review)" <>
Subject [Impala-ASF-CR] DRAFT - IMPALA-5498: Support for partial sorts
Date Fri, 23 Jun 2017 19:49:51 GMT
Thomas Tauber-Marshall has posted comments on this change.

Change subject: DRAFT - IMPALA-5498: Support for partial sorts

Patch Set 1:

File be/src/exec/

Line 104:       RETURN_IF_ERROR(sorter_->Open());
> I think you'll want to wait until the next call into GetNext() to re-open t
To be clear - what you're saying is that Open() will want to allocate memory for the new run,
but it may not be able to get it because its being used by row_batch.

So this is a perf issue, not a correctness issue, and it still makes sense to call Reset()

I'll make this change, I just want to be sure that the memory ownership transfers are happening
correctly and there aren't any potential data races.

Line 115:   do {
> I think it would make sense to open the sorter here (if it's not already op
File be/src/exec/partial-sort-node.h:

PS1, Line 30: a single tuple
> can you clarify, this is a bit unclear

PS1, Line 34: If a merge phase was performed in the sort, sorted rows are deep copied into
            : /// the output batch.
> I thought there wouldn't need to be a merge
Whoops, forgot to update this. This whole comment is just copied from SortNode.
File common/thrift/PlanNodes.thrift:

PS1, Line 353:   2: required bool use_top_n;
> this will move to TSortType, right?

PS1, Line 355: 3: optional i64 offset
> I don't think we'll need to implement the behavior to support this for part
Its also used with total sorts - we make a cost based decision about whether to execute a
sort with a limit as a top-n (implemented as a heap) or with a total sort that then only outputs
n rows.

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: Ieec2a15a0cc5240b1c13682067ab64670d1e0a38
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Thomas Tauber-Marshall <>
Gerrit-Reviewer: Matthew Jacobs <>
Gerrit-Reviewer: Thomas Tauber-Marshall <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-HasComments: Yes

View raw message