impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe McDonnell (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5039: Fix variability in parquet dictionary filtering test
Date Tue, 07 Mar 2017 22:18:24 GMT
Joe McDonnell has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/6301

Change subject: IMPALA-5039: Fix variability in parquet dictionary filtering test
......................................................................

IMPALA-5039: Fix variability in parquet dictionary filtering test

The tests for dictionary filtering look at how many row groups are processed and
how many are filtered by matching text in the profile. However, the number of row
groups processed and filtered by any individual fragment depends on how the work
is split and how many impalads are running. This causes variability in the test
output.

To fix this, the test now sums the row groups and dictionary filtered row groups
across all fragments and then compares it to the expected value. This eliminates
the variability. Since the testing is moving from the *parquet-filtering.test files
to python code, the *parquet-filtering.test file are no longer necessary.

Change-Id: I6b7b84d973b3ac678a24e82900f2637d569158bb
---
D testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet-filtering.test
D testdata/workloads/functional-query/queries/QueryTest/parquet-filtering.test
M tests/query_test/test_mt_dop.py
M tests/query_test/test_scanners.py
4 files changed, 100 insertions(+), 516 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/6301/1
-- 
To view, visit http://gerrit.cloudera.org:8080/6301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6b7b84d973b3ac678a24e82900f2637d569158bb
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Joe McDonnell <joemcdonnell@cloudera.com>

Mime
View raw message