spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [spark] LantaoJin commented on a change in pull request #26466: [SPARK-29839][SQL] Supporting STORED AS in CREATE TABLE LIKE
Date Wed, 20 Nov 2019 13:42:25 GMT
LantaoJin commented on a change in pull request #26466: [SPARK-29839][SQL] Supporting STORED
AS in CREATE TABLE LIKE
URL: https://github.com/apache/spark/pull/26466#discussion_r348486523
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
 ##########
 @@ -53,24 +53,31 @@ import org.apache.spark.sql.util.SchemaUtils
  * are identical to the ones defined in the source table.
  *
  * The CatalogTable attributes copied from the source table are storage(inputFormat, outputFormat,
- * serde, compressed, properties), schema, provider, partitionColumnNames, bucketSpec.
+ * serde, compressed, properties), schema, provider, partitionColumnNames, bucketSpec by
default.
  *
- * Use "CREATE TABLE t1 LIKE t2 USING file_format"
- * to specify new file format for t1 from a data source table t2.
+ * Use "CREATE TABLE t1 LIKE t2 USING file_format" to specify new provider for t1.
+ * For Hive compatibility, use "CREATE TABLE t1 LIKE t2 STORED AS hiveFormat"
+ * to specify new file storage format (inputFormat, outputFormat, serde) for t1.
  *
  * The syntax of using this command in SQL is:
  * {{{
  *   CREATE TABLE [IF NOT EXISTS] [db_name.]table_name
- *   LIKE [other_db_name.]existing_table_name [USING provider] [locationSpec]
+ *   LIKE [other_db_name.]existing_table_name [USING provider | STORED AS hiveFormat]
+ *   [locationSpec] [TBLPROPERTIES (property_name=property_value, ...)]
  * }}}
  */
 case class CreateTableLikeCommand(
     targetTable: TableIdentifier,
     sourceTable: TableIdentifier,
     provider: Option[String],
+    hiveFormat: Option[CatalogStorageFormat],
     location: Option[String],
 
 Review comment:
   For example, if the `sourceTableDesc.storage` is ORC format. And the CTL command contains
a LOCATION but without a STORED AS part. So the `CatalogStorageFormat.empty.copy(locationUri
= ...)` will passes to `CreateTableLikeCommand`. In this case we should use the same format
as source table's (ORC). So code will change to
   ```scala
       val newStorage = sourceTableDesc.storage.copy(
           locationUri = if (fileFormat.locationUri.isDefined) {
             fileFormat.locationUri
           } else {
             sourceTableDesc.storage.locationUri
           },
           inputFormat = if (fileFormat.inputFormat.isDefined) {
             fileFormat.inputFormat
           } else {
             sourceTableDesc.storage.inputFormat
           },
           outputFormat = if (fileFormat.outputFormat.isDefined) {
             fileFormat.outputFormat
           } else {
             sourceTableDesc.storage.outputFormat
           },
           serde = if (fileFormat.serde.isDefined) {
             fileFormat.serde
           } else {
             sourceTableDesc.storage.serde
           })
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message