impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Skye Wanderman-Milne (Code Review)" <>
Subject [Impala-CR](cdh5-trunk) IMPALA-3092: Avro table un-queriable after add columns without a default value
Date Tue, 01 Mar 2016 03:12:02 GMT
Skye Wanderman-Milne has posted comments on this change.

Change subject: IMPALA-3092: Avro table un-queriable after add columns without a default value

Patch Set 1:

Commit Message:

Line 8: without a default value
Instead of copying the JIRA title, describe what this patch actually does. e.g.

IMPALA-3092: return NULL for Avro fields missing defaults

Line 11: ****(can someone help fill this?) slots. With this patch
We used to require that any missing fields missing in an Avro file have a corresponding default
value in the table schema.
File be/src/exec/

Line 217:             "WriteDefaultValue() doesn't support default records yet, should have
You'll hit this DCHECK if there's a missing record with a default value, since WriteDefaultValue()
will return without an error.

I would add the following above the WriteDefaultValue() call:

if (is_avro_complex_type(table_record->schema)) {
    field_name, avro_type_name(table_record->schema->type));

We should add a test case for this if we don't already.

Line 252:   if (!default_value) {
if (default_value == NULL)

Line 253:     RawValue::Write(NULL, avro_header_->template_tuple, slot_desc, NULL);
We usually do template_tuple->SetNull(slot_desc->null_indicator_offset()) directly
File tests/query_test/

Line 12: class TestAvroAddColumn(ImpalaTestSuite):
Instead of creating a new test class, let's add to query_test/
Check out $IMPALA_HOME/testdata/avro_schema_resolution. In create_table.sql, the schema_resolution_test
table defines default values for every column, and then contains a file with extra columns
and file with missing columns. You should add a new similar table that's the same except it
doesn't contain any default values, and then add queries to avro-schema-resolution.test that
select from this table. Please add a record field to this table as well.

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie86421fc3da51086e566998af317faa62ba9789b
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Huaisi Xu <>
Gerrit-Reviewer: Skye Wanderman-Milne <>
Gerrit-HasComments: Yes

View raw message