impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3376: Extra definition level when writing Parquet files
Date Mon, 18 Jul 2016 15:39:37 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-3376: Extra definition level when writing Parquet files
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/3556/2/be/src/util/parquet-reader.cc
File be/src/util/parquet-reader.cc:

Line 149: void CheckDataPage(const ColumnChunk& col, const PageHeader& header, uint8_t*
data) {
> This looks odd to me, I think C++ requires constant array sizes for arrays 
Just saw this. Thought I'd comment since I have some additional info. The C++ standard doesn't
allow variable-length arrays on the stack, but both gcc and clang support it. We use this
in various places and compile with -Wno-vla to suppress warnings. Arguably we should avoid
doing this, but it's useful to be able to allocate small amounts of working space without
the overhead of malloc().

It's a bad idea though to allocate arrays on the stack if you don't have a fixed maximum size.
E.g. if uncompressed_page_size is large, we could easily overflow the stack.


-- 
To view, visit http://gerrit.cloudera.org:8080/3556
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2cafd7ef6b607ce6f815072b8af7395a892704d9
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Thomas Tauber-Marshall <tmarshall@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tmarshall@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message