impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-2.6.0_5.8.0) IMPALA-3332: Free local allocations in sorter.
Date Wed, 18 May 2016 22:12:06 GMT
Hello Tim Armstrong,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/3116

to look at the new patch set (#5).

Change subject: IMPALA-3332: Free local allocations in sorter.
......................................................................

IMPALA-3332: Free local allocations in sorter.

Sorter can have runaway memory consumption as it never frees
local allocations made in comparator_.Less(). In addition, it
doesn't check for errors generated during expression evaluation
so it may keep sorting even after failures have occurred.

This change fixes the problem by freeing local allocations for
every n invocations of comparator_.Less() where n is the row
batch size specified in the query options. Various error checks
are also added to return early if any error is encountered.

Change-Id: I941729b4836e5dbb827d4313a0b45bc5df2fa8e1
---
M be/src/runtime/sorted-run-merger.cc
M be/src/runtime/sorter.cc
M be/src/util/tuple-row-compare.h
M testdata/workloads/functional-query/queries/QueryTest/spilling.test
4 files changed, 91 insertions(+), 52 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/16/3116/5
-- 
To view, visit http://gerrit.cloudera.org:8080/3116
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I941729b4836e5dbb827d4313a0b45bc5df2fa8e1
Gerrit-PatchSet: 5
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message