From reviews-return-973035-archive-asf-public=cust-asf.ponee.io@spark.apache.org Wed Nov 20 13:42:26 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 421DF1804BB for ; Wed, 20 Nov 2019 14:42:26 +0100 (CET) Received: (qmail 21854 invoked by uid 500); 20 Nov 2019 13:42:25 -0000 Mailing-List: contact reviews-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@spark.apache.org Received: (qmail 21841 invoked by uid 99); 20 Nov 2019 13:42:25 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Nov 2019 13:42:25 +0000 From: GitBox To: reviews@spark.apache.org Subject: [GitHub] [spark] LantaoJin commented on a change in pull request #26466: [SPARK-29839][SQL] Supporting STORED AS in CREATE TABLE LIKE Message-ID: <157425734553.30444.8551367088271217505.gitbox@gitbox.apache.org> Date: Wed, 20 Nov 2019 13:42:25 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit LantaoJin commented on a change in pull request #26466: [SPARK-29839][SQL] Supporting STORED AS in CREATE TABLE LIKE URL: https://github.com/apache/spark/pull/26466#discussion_r348486523 ########## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ########## @@ -53,24 +53,31 @@ import org.apache.spark.sql.util.SchemaUtils * are identical to the ones defined in the source table. * * The CatalogTable attributes copied from the source table are storage(inputFormat, outputFormat, - * serde, compressed, properties), schema, provider, partitionColumnNames, bucketSpec. + * serde, compressed, properties), schema, provider, partitionColumnNames, bucketSpec by default. * - * Use "CREATE TABLE t1 LIKE t2 USING file_format" - * to specify new file format for t1 from a data source table t2. + * Use "CREATE TABLE t1 LIKE t2 USING file_format" to specify new provider for t1. + * For Hive compatibility, use "CREATE TABLE t1 LIKE t2 STORED AS hiveFormat" + * to specify new file storage format (inputFormat, outputFormat, serde) for t1. * * The syntax of using this command in SQL is: * {{{ * CREATE TABLE [IF NOT EXISTS] [db_name.]table_name - * LIKE [other_db_name.]existing_table_name [USING provider] [locationSpec] + * LIKE [other_db_name.]existing_table_name [USING provider | STORED AS hiveFormat] + * [locationSpec] [TBLPROPERTIES (property_name=property_value, ...)] * }}} */ case class CreateTableLikeCommand( targetTable: TableIdentifier, sourceTable: TableIdentifier, provider: Option[String], + hiveFormat: Option[CatalogStorageFormat], location: Option[String], Review comment: For example, if the `sourceTableDesc.storage` is ORC format. And the CTL command contains a LOCATION but without a STORED AS part. So the `CatalogStorageFormat.empty.copy(locationUri = ...)` will passes to `CreateTableLikeCommand`. In this case we should use the same format as source table's (ORC). So code will change to ```scala val newStorage = sourceTableDesc.storage.copy( locationUri = if (fileFormat.locationUri.isDefined) { fileFormat.locationUri } else { sourceTableDesc.storage.locationUri }, inputFormat = if (fileFormat.inputFormat.isDefined) { fileFormat.inputFormat } else { sourceTableDesc.storage.inputFormat }, outputFormat = if (fileFormat.outputFormat.isDefined) { fileFormat.outputFormat } else { sourceTableDesc.storage.outputFormat }, serde = if (fileFormat.serde.isDefined) { fileFormat.serde } else { sourceTableDesc.storage.serde }) ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org For additional commands, e-mail: reviews-help@spark.apache.org