impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Casey Ching (Code Review)" <>
Subject [Impala-CR](cdh5-trunk) Simplify creating external Kudu tables
Date Fri, 25 Mar 2016 23:49:32 GMT
Casey Ching has uploaded a new patch set (#2).

Change subject: Simplify creating external Kudu tables

Simplify creating external Kudu tables

Creating an external Kudu table was a lot harder than it needed to be
because the columns needed to be specified twice, once in Kudu and once
again in Impala. Also table properties needed to be specified and that
is more tedious than it needs to be.

  1) Read table schema from Kudu. When attempting to create an external
     table "foo" in database "bar", Impala will search for a Kudu table
     name "" and "bar" (Kudu doesn't have database name spaces
  2) The Kudu table is now required to exist at the time of creation in
  3) Disallow table properties that could conflict with an existing
     table. Ex: key_columns cannot be specified.
  4) Add KUDU as a file format.
  5) Add a startup flag to catalogd to specify the default Kudu master
     addresses. The flag is used as the default value for the table
     property kudu_master_addresses.
  6) Add the python Kudu module to the virtualenv. Building the
     virtualenv is much slower now because Cython and numpy are
     required. To help with the rebuild time --no-cache was removed.
     That option was added to help when using the dev version of impyla,
     the version number would be the same but the module contents were
     different and the cache used the old module contents.

Now an external Kudu table can be created with "CREATE EXTERNAL TABLE t
STORED AS KUDU" assuming table "t" exists in Kudu. Users can still
override table properties such as the Kudu table name or master
addresses in the usual way.

Change-Id: Ic141102818b6dad3016181b179a14024d0ff709d
M be/src/catalog/
M bin/impala-ipython
M bin/impala-py.test
M bin/impala-python
M bin/
M bin/
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/com/cloudera/impala/analysis/
M fe/src/main/java/com/cloudera/impala/analysis/
M fe/src/main/java/com/cloudera/impala/analysis/
M fe/src/main/java/com/cloudera/impala/catalog/
M fe/src/main/java/com/cloudera/impala/catalog/
M fe/src/main/java/com/cloudera/impala/catalog/delegates/
M fe/src/main/java/com/cloudera/impala/service/
M fe/src/main/java/com/cloudera/impala/service/
A fe/src/main/java/com/cloudera/impala/util/
M fe/src/main/java/com/cloudera/impala/util/
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/com/cloudera/impala/analysis/
M fe/src/test/java/com/cloudera/impala/analysis/
M fe/src/test/java/com/cloudera/impala/testutil/
M infra/python/
M infra/python/deps/requirements.txt
M testdata/bin/
M testdata/datasets/functional/functional_schema_template.sql
M testdata/workloads/functional-query/queries/QueryTest/create_kudu.test
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu-show-create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_crud.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test
M tests/common/
A tests/common/
M tests/
A tests/custom_cluster/
M tests/query_test/
38 files changed, 1,263 insertions(+), 458 deletions(-)

  git pull ssh:// refs/changes/17/2617/2
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic141102818b6dad3016181b179a14024d0ff709d
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Casey Ching <>
Gerrit-Reviewer: Matthew Jacobs <>

View raw message