impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Volker (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3776: fix 'describe formatted' for Avro tables
Date Thu, 23 Jun 2016 21:14:28 GMT
Lars Volker has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/3474

Change subject: IMPALA-3776: fix 'describe formatted' for Avro tables
......................................................................

IMPALA-3776: fix 'describe formatted' for Avro tables

For Avro tables the column information in the underlying database of the
Hive metastore can be different from what is specified in the
'avro.schema.url' file. HIVE-6308 aimed to improve upon this, but for
older tables the two don't necessarily align.

There are two possible cases:

1) Hive's underlying database contains a column, which is not present in
the Avro schema file. In this case we encounter a NullPointerException
in DescribeResultFactory.java#L189 when trying to lookup the column in
the internal table object.

2) The avro schema contains a column, which is not present in the
underlying database. In this case the column will not be displayed in
describe formatted.

I don't know how to automatically test this, but I verified this
manually by creating an Avro table with an external schema file in Hive.
This populated the underlying database with the column information. I
then either removed a column from the Avro schema file (case 1) or
cleared the column information from the "COLUMNS_V2" table in the
underlying database (case 2) and verified that the change fixed both
cases.

Change-Id: Ic640f18acf7a1731f34b22c50ebbb462dfee78bd
---
M fe/src/main/java/com/cloudera/impala/catalog/Column.java
M fe/src/main/java/com/cloudera/impala/catalog/Table.java
M fe/src/main/java/com/cloudera/impala/service/DescribeResultFactory.java
3 files changed, 29 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/74/3474/1
-- 
To view, visit http://gerrit.cloudera.org:8080/3474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic640f18acf7a1731f34b22c50ebbb462dfee78bd
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Lars Volker <lv@cloudera.com>

Mime
View raw message