impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anurag Mantripragada (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-9070: Include table location in lineage for 'CREATE EXTERNAL TABLE' DDL.
Date Sat, 19 Oct 2019 00:39:30 GMT
Anurag Mantripragada has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14515


Change subject: IMPALA-9070: Include table location in lineage for 'CREATE EXTERNAL TABLE'
DDL.
......................................................................

IMPALA-9070: Include table location in lineage for 'CREATE EXTERNAL
TABLE' DDL.

Atlas needs table location to establish lineage between a newly
created external table and its table location.

The table location information is not available until the createTable
catalog op succeeds. After this change, location information is sent
to the backend in the TDDLExecResponse message which adds it to the
lineage graph. This information is sent only for create external
table queries.

Testing:
Added a test to verify the tableLocation field is populated for a
create external table query lineage. Also, modified the
lineage.test file to include location information for all lineages.

Change-Id: If02b0cc16d52c1956298171628f5737cab62ce9f
---
M be/src/service/client-request-state.cc
M be/src/util/lineage-util.h
M common/thrift/CatalogService.thrift
M common/thrift/LineageGraph.thrift
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M testdata/workloads/functional-query/queries/QueryTest/lineage.test
M tests/custom_cluster/test_lineage.py
7 files changed, 920 insertions(+), 449 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/15/14515/1
-- 
To view, visit http://gerrit.cloudera.org:8080/14515
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If02b0cc16d52c1956298171628f5737cab62ce9f
Gerrit-Change-Number: 14515
Gerrit-PatchSet: 1
Gerrit-Owner: Anurag Mantripragada <anurag@cloudera.com>

Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message