impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2399: Check for mem limit in allocations in parquet scanner and decompressor
Date Sun, 06 Mar 2016 19:30:16 GMT
Hello Tim Armstrong,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/2203

to look at the new patch set (#13).

Change subject: IMPALA-2399: Check for mem limit in allocations in parquet scanner and decompressor
......................................................................

IMPALA-2399: Check for mem limit in allocations in parquet scanner and decompressor

This change replaces all calls to MemPool::Allocate() with
MemPool::TryAllocate() in the parquet scanner and the decompressor.
Also streamline CheckQueryState() to avoid unnecessary spinlock
acquisition for the common case when there is no error. Also
removes some dead code in the text converter.

MemPool::Allocate() is also updated to return a valid pointer
instead of NULL when the allocation size is zero. NULL is only
returned during allocation failure.

Change-Id: Ic70400407b7662999332448f4d1bce2cc344ca89
---
M be/src/exec/base-sequence-scanner.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-scanner.h
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/text-converter.cc
M be/src/exec/text-converter.h
M be/src/runtime/mem-pool.cc
M be/src/runtime/mem-pool.h
M be/src/runtime/mem-tracker.cc
M be/src/runtime/mem-tracker.h
M be/src/runtime/runtime-state.cc
M be/src/runtime/runtime-state.h
M be/src/util/codec.h
M be/src/util/decompress.cc
M be/src/util/runtime-profile.h
17 files changed, 253 insertions(+), 119 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/03/2203/13
-- 
To view, visit http://gerrit.cloudera.org:8080/2203
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic70400407b7662999332448f4d1bce2cc344ca89
Gerrit-PatchSet: 13
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Skye Wanderman-Milne <skye@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message