impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe McDonnell (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5525 Extend TestScannersFuzzing to test uncompressed parquet
Date Fri, 15 Sep 2017 16:49:26 GMT
Joe McDonnell has posted comments on this change.

Change subject: IMPALA-5525 Extend TestScannersFuzzing to test uncompressed parquet
......................................................................


Patch Set 2:

(2 comments)

Take a look at these suggestions and let me know if they make sense.

http://gerrit.cloudera.org:8080/#/c/8056/2/tests/query_test/test_scanners_fuzz.py
File tests/query_test/test_scanners_fuzz.py:

PS2, Line 97: fq_tbl_name = "functional_parquet" + "." + tbl_name
Do not create a table in the default schemas. Instead, with the changes I described elsewhere,
create a separate clone function that will create an uncompressed parquet table directly in
unique_database and then pass that into run_fuzz_test.

The existing tests will run a simple table clone function that is equivalent to the current
code.


PS2, Line 129:     self.execute_query("create table %s.%s like %s" % (unique_database, table,
table))
             :     fuzz_table_location = get_fs_path("/test-warehouse/{0}.db/{1}".format(
             :         unique_database, table))
Pull this logic out into its own function e.g. clone_table. run_fuzz_test should take in a
table that has already been created in unique_database. It should not do the clone itself.
This allows you to use a different clone function to create a parquet table without compression.


-- 
To view, visit http://gerrit.cloudera.org:8080/8056
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I760de7203a51cf82b16016fa8043cadc7c8325bc
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Pranay Singh
Gerrit-Reviewer: Joe McDonnell <joemcdonnell@cloudera.com>
Gerrit-Reviewer: Pranay Singh
Gerrit-HasComments: Yes

Mime
View raw message