impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Skye Wanderman-Milne (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3441, IMPALA-3659: check for malformed Avro data
Date Thu, 09 Jun 2016 20:45:28 GMT
Skye Wanderman-Milne has posted comments on this change.

Change subject: IMPALA-3441, IMPALA-3659: check for malformed Avro data
......................................................................


Patch Set 12:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/3072/12/be/src/exec/hdfs-avro-scanner.cc
File be/src/exec/hdfs-avro-scanner.cc:

Line 123:   if (num_entries < 1) return Status("File header metadata has no data");
> Can we add an error code for this and the other similar check. Something ge
Done


http://gerrit.cloudera.org:8080/#/c/3072/12/be/src/exec/read-write-util.cc
File be/src/exec/read-write-util.cc:

Line 46:     shift += 7;
> I'm pretty sure this can overflow. E.g. if all bytes before 0x80 are 0x7f. 
Done, as per your suggestion. I have this case in the unit test too but don't check for a
particular result.


http://gerrit.cloudera.org:8080/#/c/3072/12/be/src/exec/read-write-util.h
File be/src/exec/read-write-util.h:

Line 110:   /// (i.e. the first invalid byte).
> Document the behaviour in error cases. If the zig-zag encoded number has to
Done


http://gerrit.cloudera.org:8080/#/c/3072/12/be/src/exec/scanner-context.cc
File be/src/exec/scanner-context.cc:

Line 128:     DCHECK(!completed_io_buffers_.empty());
> I'm not sure I understand this DCHECK. It seems like we could maybe hit it 
I put it just to have some extra validation that the condition correctly specifies that we're
at eof, since I don't think we create scan ranges for empty files. I'll remove it since it's
confusing though.


-- 
To view, visit http://gerrit.cloudera.org:8080/3072
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I801a11c496a128e02c564c2a9c44baa5a97be132
Gerrit-PatchSet: 12
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Skye Wanderman-Milne <skye@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Skye Wanderman-Milne <skye@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message