impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Internal Jenkins (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3105: avoid overrunning allocated tuple buffer
Date Fri, 15 Apr 2016 00:59:56 GMT
Internal Jenkins has submitted this change and it was merged.

Change subject: IMPALA-3105: avoid overrunning allocated tuple buffer
......................................................................


IMPALA-3105: avoid overrunning allocated tuple buffer

This change reworks allocation of tuple buffers. Previously the code
tried to estimate via MaxTupleBufferSize() the maximum size of buffer it
would need to allocate, assuming that AT_CAPACITY_MEM_USAGE was not
exceeded. However, AT_CAPACITY_MEM_USAGE is a soft limit so can be
exceeded.

The fix is to change the code to set the batch's 'capacity_', which is a
hard limit, to a value such that FIXED_LEN_BUFFER_LIMIT <=
AT_CAPACITY_MEM_USAGE won't be exceeded.

Change-Id: Idfd9cd681875821c1c379d97586d3f4850aae622
Reviewed-on: http://gerrit.cloudera.org:8080/2473
Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
Tested-by: Internal Jenkins
---
M be/src/exec/data-source-scan-node.cc
M be/src/exec/data-source-scan-node.h
M be/src/exec/hbase-scan-node.cc
M be/src/exec/hbase-scan-node.h
M be/src/exec/hdfs-scanner.cc
M be/src/exec/kudu-scanner.cc
M be/src/exec/union-node.cc
M be/src/runtime/row-batch.cc
M be/src/runtime/row-batch.h
9 files changed, 77 insertions(+), 70 deletions(-)

Approvals:
  Internal Jenkins: Verified
  Tim Armstrong: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/2473
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Idfd9cd681875821c1c379d97586d3f4850aae622
Gerrit-PatchSet: 10
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Casey Ching <casey@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message