spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From viirya <...@git.apache.org>
Subject [GitHub] spark pull request #22313: [SPARK-25306][SQL] Avoid skewed filter trees to s...
Date Tue, 04 Sep 2018 05:03:52 GMT
Github user viirya commented on a diff in the pull request:

    https://github.com/apache/spark/pull/22313#discussion_r214787227
  
    --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala
---
    @@ -398,6 +398,24 @@ class FilterPushdownBenchmark extends SparkFunSuite with BenchmarkBeforeAndAfter
           }
         }
       }
    +
    +  test(s"Pushdown benchmark with many filters") {
    +    val numRows = 1
    +    val width = 500
    +
    +    withTempPath { dir =>
    +      val columns = (1 to width).map(i => s"id c$i")
    +      val df = spark.range(1).selectExpr(columns: _*)
    +      withTempTable("orcTable", "patquetTable") {
    --- End diff --
    
    nit: a typo, `patquetTable`.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message