impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharath Vissapragada (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-2.6.0 5.8.0) CDH-41511: Fix the preconditions check in HdfsTable#setAvroSchema()
Date Wed, 22 Jun 2016 16:29:11 GMT
Bharath Vissapragada has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/3446

Change subject: CDH-41511: Fix the preconditions check in HdfsTable#setAvroSchema()
......................................................................

CDH-41511: Fix the preconditions check in HdfsTable#setAvroSchema()

In HdfsTable#setAvroSchema(), we rely on an incorrect precondition
check(!nonPartFieldSchemas_.isEmpty()) to make sure loadSchema() is
called. However nonPartFieldSchemas_ can still be empty for Avro
tables created using older versions of Hive. We take care of this
in the schema reconciliation step where we read the Avro schema to
populate the column list. This incorrect check prevents the Catalog
from reaching the schema reconciliation code path.

Fix: Added a new flag isSchemaLoaded_ and setAvroSchema() uses it
to check if the schema has been loaded properly.

Testing: I couldn't reproduce the issue directly by creating Avro
tables without any columns. So, I manually deleted the entries from
"COLUMNS_V2" table in the hms db to make sure to hit the incorrect
preconditions check. Applying this patch fixed it and Impala loaded
the table correctly and I can query it now.

Change-Id: I02b6852385d4f4b470f763308817991e60557060
---
M fe/src/main/java/com/cloudera/impala/catalog/HdfsTable.java
1 file changed, 6 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/46/3446/1
-- 
To view, visit http://gerrit.cloudera.org:8080/3446
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I02b6852385d4f4b470f763308817991e60557060
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Bharath Vissapragada <bharathv@cloudera.com>

Mime
View raw message