impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-2.6.0_5.8.0) IMPALA-3332: Free local allocations in sorter.
Date Wed, 18 May 2016 17:59:33 GMT
Michael Ho has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/3116

Change subject: IMPALA-3332: Free local allocations in sorter.
......................................................................

IMPALA-3332: Free local allocations in sorter.

Sorter can have runaway memory consumption as it never frees
local allocations made in comparator_.Less(). In addition, it
doesn't check for errors generated during expression evaluation
so it may keep sorting even after failures have occurred.

This change fixes the problem by freeing local allocations for
every n invocations of comparator_.Less() where n is the row
batch size specified in the query options. Various error checks
are also added to return early if any error is encountered.

Change-Id: I941729b4836e5dbb827d4313a0b45bc5df2fa8e1
---
M be/src/runtime/sorter.cc
M be/src/util/tuple-row-compare.h
M testdata/workloads/functional-query/queries/QueryTest/spilling.test
3 files changed, 84 insertions(+), 50 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/16/3116/1
-- 
To view, visit http://gerrit.cloudera.org:8080/3116
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I941729b4836e5dbb827d4313a0b45bc5df2fa8e1
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Michael Ho <kwho@cloudera.com>

Mime
View raw message