spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gatorsmile <...@git.apache.org>
Subject [GitHub] spark pull request #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmar...
Date Wed, 30 May 2018 00:03:54 GMT
Github user gatorsmile commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21288#discussion_r191610297
  
    --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala
---
    @@ -131,211 +132,214 @@ object FilterPushdownBenchmark {
         }
     
         /*
    +    OpenJDK 64-Bit Server VM 1.8.0_171-b10 on Linux 4.14.26-46.32.amzn1.x86_64
         Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
         Select 0 string row (value IS NULL):     Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)
  Relative
         ------------------------------------------------------------------------------------------------
    -    Parquet Vectorized                            8452 / 8504          1.9         537.3
      1.0X
    -    Parquet Vectorized (Pushdown)                  274 /  281         57.3          17.4
     30.8X
    -    Native ORC Vectorized                         8167 / 8185          1.9         519.3
      1.0X
    -    Native ORC Vectorized (Pushdown)               365 /  379         43.1          23.2
     23.1X
    +    Parquet Vectorized                            2961 / 3123          5.3         188.3
      1.0X
    +    Parquet Vectorized (Pushdown)                 3057 / 3121          5.1         194.4
      1.0X
    --- End diff --
    
    I have not tried it yet, but is it related to the recent change we made in the parquet
reader?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message