spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hvanhov...@apache.org
Subject spark git commit: Remove non-obvious conf settings from TPCDS benchmark
Date Fri, 17 Jun 2016 16:47:59 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-2.0 3c3865d0b -> de964e419


Remove non-obvious conf settings from TPCDS benchmark

## What changes were proposed in this pull request?

My fault -- these 2 conf entries are mysteriously hidden inside the benchmark code and makes
it non-obvious to disable whole stage codegen and/or the vectorized parquet reader.

PS: Didn't attach a JIRA as this change should otherwise be a no-op (both these conf are enabled
by default in Spark)

## How was this patch tested?

N/A

Author: Sameer Agarwal <sameer@databricks.com>

Closes #13726 from sameeragarwal/tpcds-conf.

(cherry picked from commit 34d6c4cd113729fcc1d0bc1df8916d06b8854922)
Signed-off-by: Herman van Hovell <hvanhovell@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/de964e41
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/de964e41
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/de964e41

Branch: refs/heads/branch-2.0
Commit: de964e4194cac1ad9ed4ca7f16dbb809fc7c9471
Parents: 3c3865d
Author: Sameer Agarwal <sameer@databricks.com>
Authored: Fri Jun 17 09:47:41 2016 -0700
Committer: Herman van Hovell <hvanhovell@databricks.com>
Committed: Fri Jun 17 09:47:55 2016 -0700

----------------------------------------------------------------------
 .../apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala | 2 --
 1 file changed, 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/de964e41/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
index 398d8d9..957a1d6 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
@@ -64,8 +64,6 @@ object TPCDSQueryBenchmark {
     require(dataLocation.nonEmpty,
       "please modify the value of dataLocation to point to your local TPCDS data")
     val tableSizes = setupTables(dataLocation)
-    spark.conf.set(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key, "true")
-    spark.conf.set(SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key, "true")
     queries.foreach { name =>
       val queryString = fileToString(new File(Thread.currentThread().getContextClassLoader
         .getResource(s"tpcds/$name.sql").getFile))


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message