impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5419: Check for cancellation when building hash tables
Date Thu, 01 Jun 2017 23:57:06 GMT
Michael Ho has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/7047

Change subject: IMPALA-5419: Check for cancellation when building hash tables
......................................................................

IMPALA-5419: Check for cancellation when building hash tables

When inserting build rows into the hash table for each partition
in PhjBuilder::Partition::BuildHashTable(), we didn't check for
cancellation. This may lead to orphaned fragments running for
excessive amount of time and consuming a lot of memory after the
query has been cancelled.

This change fixes the problem by checking for cancellation per
row batch in the loop inside PhjBuilder::Partition::BuildHashTable().

Manually verified with the following query that cancelling the query
right after all build rows have been returned by the scan node will
not cause memory usage to continue going up.

select /* +straight_join */ count(*)
from tpch20_parquet.lineitem t1
  join /* +broadcast */ tpch20_parquet.lineitem t2
  on t1.l_orderkey = t2.l_orderkey

Change-Id: I8047f532f55dc0118f7a843c91275f752c8a190d
---
M be/src/exec/partitioned-hash-join-builder.cc
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/7047/1
-- 
To view, visit http://gerrit.cloudera.org:8080/7047
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8047f532f55dc0118f7a843c91275f752c8a190d
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Ho <kwho@cloudera.com>

Mime
View raw message