spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HyukjinKwon <...@git.apache.org>
Subject [GitHub] spark pull request #19464: [SPARK-22233] [core] Allow user to filter out emp...
Date Fri, 13 Oct 2017 17:49:03 GMT
Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19464#discussion_r144617310
  
    --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
    @@ -510,4 +510,87 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {
         }
       }
     
    +  test("spark.files.ignoreEmptySplits work correctly (old Hadoop API)") {
    +    val conf = new SparkConf()
    +    conf.setAppName("test").setMaster("local").set(IGNORE_EMPTY_SPLITS, true)
    +    sc = new SparkContext(conf)
    +
    +    def testIgnoreEmptySplits(
    +      data: Array[Tuple2[String, String]],
    +      actualPartitionNum: Int,
    +      expectedPart: String,
    +      expectedPartitionNum: Int): Unit = {
    --- End diff --
    
    Indentation..
    
    ```scala
    def testIgnoreEmptySplits(
        data: Array[Tuple2[String, String]],
        ...
        expectedPartitionNum: Int): Unit = {
      val output = new File(tempDir, "output")
      ...
    ```


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message