impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe McDonnell (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-6068: Scale back fixing functional-types
Date Mon, 04 Dec 2017 17:29:11 GMT
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/8690 )

Change subject: IMPALA-6068: Scale back fixing functional-types
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8690/4/testdata/datasets/functional/functional_schema_template.sql
File testdata/datasets/functional/functional_schema_template.sql:

http://gerrit.cloudera.org:8080/#/c/8690/4/testdata/datasets/functional/functional_schema_template.sql@a2115
PS4, Line 2115: 
> What the heck is this actually doing?  If I am reading this correctly, it i
The LOAD section populates the text version of the table (functional.table_with_header_2).
In this case, a csv file where the first line is column names. The DEPENDENT_LOAD populates
other encodings. This table is very hacky. The only encoding other than text is gzip encoded
text (the file that determines which formats apply for which tables is testdata/datasets/functional/schema_constraints.csv).
We want to preserve the fact that the first line is column names. I don't think Hive would
preserve this if we were to create a gzip table and do an "insert as select * from table_with_header_2".
So, we load a gzip compressed version of the same file to populate functional_gzip.table_with_header_2.
This only works because the only non-text format is gzip compressed text.



-- 
To view, visit http://gerrit.cloudera.org:8080/8690
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6c242cca209a7138b10ad517076707709b5cd204
Gerrit-Change-Number: 8690
Gerrit-PatchSet: 4
Gerrit-Owner: Zach Amsden <zamsden@cloudera.com>
Gerrit-Reviewer: David Knupp <dknupp@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell <joemcdonnell@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <philip@cloudera.com>
Gerrit-Reviewer: Zach Amsden <zamsden@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Dec 2017 17:29:11 +0000
Gerrit-HasComments: Yes

Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message