spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HyukjinKwon <...@git.apache.org>
Subject [GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...
Date Wed, 25 Jul 2018 06:00:27 GMT
Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20949#discussion_r204988168
  
    --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
    @@ -514,6 +516,41 @@ class CSVSuite extends QueryTest with SharedSQLContext with SQLTestUtils
with Te
         }
       }
     
    +  test("SPARK-19018: Save csv with custom charset") {
    +
    +    // scalastyle:off nonascii
    +    val content = "µß áâä ÁÂÄ"
    +    // scalastyle:on nonascii
    +
    +    Seq("iso-8859-1", "utf-8", "utf-16", "utf-32", "windows-1250").foreach { encoding
=>
    +      withTempPath { path =>
    +        val csvDir = new File(path, "csv")
    +        Seq(content).toDF().write
    +          .option("encoding", encoding)
    +          .csv(csvDir.getCanonicalPath)
    +
    +        csvDir.listFiles().filter(_.getName.endsWith("csv")).foreach({ csvFile =>
    --- End diff --
    
    nit: `.foreach({` -> `.foreach {` per https://github.com/databricks/scala-style-guide#anonymous-methods


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message