impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2399: Check for mem limit in allocations in parquet scanner and decompressor
Date Wed, 09 Mar 2016 20:27:39 GMT
Michael Ho has posted comments on this change.

Change subject: IMPALA-2399: Check for mem limit in allocations in parquet scanner and decompressor
......................................................................


Patch Set 14:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/2203/14/be/src/exec/base-sequence-scanner.cc
File be/src/exec/base-sequence-scanner.cc:

Line 95:   DCHECK(prepared_ == (batch_ != NULL));
> That's true. We can only rely on if(batch_ != NULL) and DCHECK(batch_ != NU
Actually, prepared_ becomes kind of useless now so it's removed. Instead, when batch_ == NULL,
DCHECK no resources are leaked (e.g. mempools and completed_io_buffers_ are empty).


http://gerrit.cloudera.org:8080/#/c/2203/14/be/src/exec/hdfs-parquet-scanner.cc
File be/src/exec/hdfs-parquet-scanner.cc:

Line 1648:         DCHECK_EQ(num_rows, 0);
> Why is this DCHECK interesting? i.e. why does the caller care about num_row
DCHECK removed.


http://gerrit.cloudera.org:8080/#/c/2203/14/be/src/runtime/collection-value.h
File be/src/runtime/collection-value.h:

Line 32:   int64_t num_tuples;
> I think the frontend needs to know about this change. Maybe revert this cha
Thanks for pointing that out. I reverted this line of change and some others in CollectionValueBuilder
 but updated ByteSIze() to cast num_tuples to int64_t (which I believe won't break the interface
with frontend). Also, keep the removal of the size restriction of allocations in CollectionValueBuilder.


-- 
To view, visit http://gerrit.cloudera.org:8080/2203
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ic70400407b7662999332448f4d1bce2cc344ca89
Gerrit-PatchSet: 14
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Skye Wanderman-Milne <skye@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message