impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3105: avoid overrunning allocated tuple buffer
Date Wed, 13 Apr 2016 20:23:47 GMT
Tim Armstrong has uploaded a new patch set (#7).

Change subject: IMPALA-3105: avoid overrunning allocated tuple buffer
......................................................................

IMPALA-3105: avoid overrunning allocated tuple buffer

This change reworks allocation of tuple buffers. Previously the code
tried to estimate via MaxTupleBufferSize() the maximum size of buffer it
would need to allocate, assuming that AT_CAPACITY_MEM_USAGE was not
exceeded. However, AT_CAPACITY_MEM_USAGE is a soft limit so can be
exceeded.

The fix is to change the code to set the batch's 'capacity_', which is a
hard limit, to a value such that FIXED_LEN_BUFFER_LIMIT <=
AT_CAPACITY_MEM_USAGE won't be exceeded.

Change-Id: Idfd9cd681875821c1c379d97586d3f4850aae622
---
M be/src/exec/data-source-scan-node.cc
M be/src/exec/data-source-scan-node.h
M be/src/exec/hbase-scan-node.cc
M be/src/exec/hbase-scan-node.h
M be/src/exec/hdfs-scanner.cc
M be/src/exec/kudu-scanner.cc
M be/src/exec/union-node.cc
M be/src/runtime/row-batch.cc
M be/src/runtime/row-batch.h
9 files changed, 77 insertions(+), 70 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/73/2473/7
-- 
To view, visit http://gerrit.cloudera.org:8080/2473
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idfd9cd681875821c1c379d97586d3f4850aae622
Gerrit-PatchSet: 7
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Casey Ching <casey@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message