spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [spark] rdblue commented on a change in pull request #27992: [SPARK-31230][SQL] use statement plans in DataFrameWriter(V2)
Date Mon, 23 Mar 2020 22:03:17 GMT
rdblue commented on a change in pull request #27992: [SPARK-31230][SQL] use statement plans
in DataFrameWriter(V2)
URL: https://github.com/apache/spark/pull/27992#discussion_r396783853
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriterV2.scala
 ##########
 @@ -120,19 +117,18 @@ final class DataFrameWriterV2[T] private[sql](table: String, ds: Dataset[T])
   }
 
   override def create(): Unit = {
-    // create and replace could alternatively create ParsedPlan statements, like
-    // `CreateTableFromDataFrameStatement(UnresolvedRelation(tableName), ...)`, to keep the
catalog
-    // resolution logic in the analyzer.
     runCommand("create") {
-      CreateTableAsSelect(
-        catalog,
-        identifier,
-        partitioning.getOrElse(Seq.empty),
+      CreateTableAsSelectStatement(
 
 Review comment:
   This no longer passes the options as write options because the statement plans don't have
write options, only table options.
   
   This change should add `writeOptions` to the statement and update conversions to pass them
through to the exec plans.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message