impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharath Vissapragada (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3314/IMPALA-3513: Fix querying tables/partitions altered to Avro format
Date Thu, 19 May 2016 10:42:50 GMT
Bharath Vissapragada has uploaded a new patch set (#2).

Change subject: IMPALA-3314/IMPALA-3513: Fix querying tables/partitions altered to Avro format
......................................................................

IMPALA-3314/IMPALA-3513: Fix querying tables/partitions altered to Avro format

Bug: Impalads crash if we query an Avro table with stale metadata

Cause: This happens because avroSchema_ is not set in HdfsTable,
which is not propagated to the avro scanner and it doesn't have
appropriate checks to make sure the schema is non-null.

The patch fixes the following.

1. Avro scanner should gracefully handle the case where the avro schema
   is not set. Appropriate null checks and a meaning error message have
   been added.

2. This is a special case with multi-fileformat partitioned tables.
   avroSchema_ should be set in HdfsTable even if any subset of the
   partitions are backed by avro. Without this patch, we only set it
   if the base table file format is Avro.

Change-Id: I09262d3a7b85a2263c721f3beafd0cab2a1bdf4b
---
M be/src/exec/hdfs-avro-scanner.cc
M be/src/exec/hdfs-avro-scanner.h
M fe/src/main/java/com/cloudera/impala/catalog/HdfsPartition.java
M fe/src/main/java/com/cloudera/impala/catalog/HdfsTable.java
A testdata/workloads/functional-query/queries/QueryTest/avro-stale-schema.test
M tests/query_test/test_avro_schema_resolution.py
6 files changed, 118 insertions(+), 19 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/45/3045/2
-- 
To view, visit http://gerrit.cloudera.org:8080/3045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I09262d3a7b85a2263c721f3beafd0cab2a1bdf4b
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bharathv@cloudera.com>

Mime
View raw message