impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3729: batch size┘coverage for avro scanner
Date Tue, 19 Jul 2016 21:07:19 GMT
Hello Taras Bobrovytsky, Internal Jenkins, Dan Hecht,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/3413

to look at the new patch set (#8).

Change subject: IMPALA-3729: batch_size=1 coverage for avro scanner
......................................................................

IMPALA-3729: batch_size=1 coverage for avro scanner

Also fix a stale comment in the avro scanner header.

The main work here is to fix the handling of empty result sets in the
test result verifier. This is a problem because we wanted to verify
that the results in the test file were a superset of the rows
returned, and this was thrown off by superflous '' rows in the expected
and actual result sets.

The basic problem is that the way test file sections
was parsed conflated an empty result section with non-empty result
section that had a single empty string. I.e.:

---- RESULTS
====

vs
---- RESULTS

====

both got resolved to [''].

Change-Id: Ia007e558d92c7e4ce30be90446fdbb1f50a0ebc4
---
M be/src/exec/hdfs-avro-scanner.h
M bin/load-data.py
M testdata/bin/generate-schema-statements.py
M testdata/workloads/functional-query/queries/DataErrorsTest/avro-errors.test
M testdata/workloads/functional-query/queries/QueryTest/load.test
M testdata/workloads/functional-query/queries/QueryTest/test-unmatched-schema.test
M tests/beeswax/impala_beeswax.py
M tests/common/test_result_verifier.py
M tests/data_errors/test_data_errors.py
M tests/unittests/test_file_parser.py
M tests/util/test_file_parser.py
11 files changed, 86 insertions(+), 41 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/13/3413/8
-- 
To view, visit http://gerrit.cloudera.org:8080/3413
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia007e558d92c7e4ce30be90446fdbb1f50a0ebc4
Gerrit-PatchSet: 8
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message