impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Volker (Code Review)" <>
Subject [Impala-CR](cdh5-trunk) IMPALA-3776: fix 'describe formatted' for Avro tables
Date Thu, 23 Jun 2016 21:14:28 GMT
Lars Volker has uploaded a new change for review.

Change subject: IMPALA-3776: fix 'describe formatted' for Avro tables

IMPALA-3776: fix 'describe formatted' for Avro tables

For Avro tables the column information in the underlying database of the
Hive metastore can be different from what is specified in the
'avro.schema.url' file. HIVE-6308 aimed to improve upon this, but for
older tables the two don't necessarily align.

There are two possible cases:

1) Hive's underlying database contains a column, which is not present in
the Avro schema file. In this case we encounter a NullPointerException
in when trying to lookup the column in
the internal table object.

2) The avro schema contains a column, which is not present in the
underlying database. In this case the column will not be displayed in
describe formatted.

I don't know how to automatically test this, but I verified this
manually by creating an Avro table with an external schema file in Hive.
This populated the underlying database with the column information. I
then either removed a column from the Avro schema file (case 1) or
cleared the column information from the "COLUMNS_V2" table in the
underlying database (case 2) and verified that the change fixed both

Change-Id: Ic640f18acf7a1731f34b22c50ebbb462dfee78bd
M fe/src/main/java/com/cloudera/impala/catalog/
M fe/src/main/java/com/cloudera/impala/catalog/
M fe/src/main/java/com/cloudera/impala/service/
3 files changed, 29 insertions(+), 13 deletions(-)

  git pull ssh:// refs/changes/74/3474/1
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic640f18acf7a1731f34b22c50ebbb462dfee78bd
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Lars Volker <>

View raw message