impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-3487: validate decimal type in Avro file schema
Date Fri, 28 Oct 2016 15:24:16 GMT
Tim Armstrong has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/4876

Change subject: IMPALA-3487: validate decimal type in Avro file schema
......................................................................

IMPALA-3487: validate decimal type in Avro file schema

This patch prevents an invalid decimal type in an Avro file schema from
crashing Impala. Most invalid Avro schemas are caught by the frontend,
but file schemas still need to be validated by the backend.

After this patch files with bad schemas are skipped.

Testing:
This was hit very rarely by the scanner fuzzing. Added a regression test that scans
a file with a bad schema.

Change-Id: I25a326ee2220bc14d3b5f887dc288b4adf859cfc
---
M be/src/exec/hdfs-avro-scanner.cc
M be/src/runtime/decimal-test.cc
M be/src/runtime/types.h
M be/src/util/avro-util.cc
M be/src/util/avro-util.h
M common/thrift/generate_error_codes.py
M testdata/bad_avro_snap/README
A testdata/bad_avro_snap/invalid_decimal_schema.avro
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/DataErrorsTest/avro-errors.test
11 files changed, 102 insertions(+), 26 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/4876/1
-- 
To view, visit http://gerrit.cloudera.org:8080/4876
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I25a326ee2220bc14d3b5f887dc288b4adf859cfc
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message