impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5412 Scan returns wrong partition-column values when scanning multiple partitions pointing to the same filesystem location.
Date Wed, 09 Aug 2017 15:59:00 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-5412 Scan returns wrong partition-column values when scanning multiple
partitions pointing to the same filesystem location.
......................................................................


Patch Set 1:

(2 comments)

Looks like MJ and I collided. I think we mostly agree here.

http://gerrit.cloudera.org:8080/#/c/7625/1/tests/metadata/test_partition_metadata.py
File tests/metadata/test_partition_metadata.py:

PS1, Line 80: avro
> does this need to be avro? seems like text would work, and this test class 
The bug affects Seq/RC/Avro only.


PS1, Line 110: note, that shortcoming on avro writer would result the inserted value as NULL,
             :     # but the point is the second column for partition i
> This isn't a clear sentence; why does the avro writer insert NULL?
The Avro writer is broken. It always represents null values as the second element of the union,
ignoring any schema. But the default avro schema in the frontend represents null values as
the first element of the union.

I guess we could flip the 0 and 1 in HdfsAvroTableWriter::AppendField() and at least make
the writer usable for this test.


-- 
To view, visit http://gerrit.cloudera.org:8080/7625
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie74b305377248045c0d87b911943e1cabb7223e9
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Gabor Kaszab <gaborkaszab@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Attila Jeges <attilaj@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <laszlo.gaal@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message