impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Taras Bobrovytsky (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4363: Add Parquet timestamp validation
Date Sat, 19 Nov 2016 05:22:25 GMT
Taras Bobrovytsky has posted comments on this change.

Change subject: IMPALA-4363: Add Parquet timestamp validation

Patch Set 6:

Commit Message:

PS6, Line 12: did't
> typo

PS6, Line 14: date
> year*
File be/src/exec/

Line 526:     parent_->parse_status_ = Status(*msg);
> the claim is we are pulling this out to avoid status construction code, but
Good point, Done.

PS6, Line 598:  
> nit: extraneous space

Line 599:   if (!src->IsValidDate()) {
Done, (also changed the order of the if statement).
File be/src/exec/parquet-column-readers.h:

PS6, Line 512: 
> was this an incorrect comment?
I saw a comment in the implementation, so I didn't think it was true any more:
"// AssembleCollection() advances child readers, so we don't need to call NextLevels()"

I looked at the implementation, but it turns out AssembleCollection() actually calls NextLevels().
I'll put the comment back.

PS6, Line 449: void* slot
> shouldn't this be 'tuple'.  but the fact that this compiled probably means 
Yeah, I don't think this is used anywhere, so I'll delete it.
File be/src/runtime/timestamp-value.h:

Line 156:     return date_.day_number() >= MIN_DAY_NUMBER && date_.day_number()
> are we sure these are the only invalidate TimestampValue's that may have be
Yes, it turns out there are (but they don't cause a crash). I added validation of time_ as
File testdata/bad_parquet_data/README:

PS6, Line 16: out-of-range-timestamp.parq
> not sure what you mean -- aren't you adding a parquet file to test this? co
I'm adding a file to testdata/data, this readme is in testdata/bad_parquet_data
File testdata/workloads/functional-query/queries/QueryTest/out-of-range-timestamp-abort-on-error.test:

PS6, Line 7: Parquet file '$NAM
> why is the message repeated?

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: I9988449aa0dc0f39fabb91ce6cce0dd8a06e8bcf
Gerrit-PatchSet: 6
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Taras Bobrovytsky <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: Dan Hecht <>
Gerrit-Reviewer: Lars Volker <>
Gerrit-Reviewer: Laszlo Gaal <>
Gerrit-Reviewer: Taras Bobrovytsky <>
Gerrit-HasComments: Yes

View raw message