spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dongjoon-hyun <...@git.apache.org>
Subject [GitHub] spark pull request #19571: [SPARK-15474][SQL] Write and read back non-emtpy ...
Date Wed, 25 Oct 2017 19:09:53 GMT
Github user dongjoon-hyun commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19571#discussion_r146957878
  
    --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala ---
    @@ -252,6 +253,13 @@ private[orc] class OrcOutputWriter(
       override def close(): Unit = {
         if (recordWriterInstantiated) {
           recordWriter.close(Reporter.NULL)
    +    } else {
    +      // SPARK-15474 Write empty orc file with correct schema
    +      val conf = context.getConfiguration()
    --- End diff --
    
    Previously, only ORC does. So, it creates more issues like SPARK-22258 (#19477) and SPARK-21762.
This is consistent with the other data sources like Parquet.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message