spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From marmb...@apache.org
Subject git commit: [SPARK-2283][SQL] Reset test environment before running PruningSuite
Date Thu, 26 Jun 2014 01:42:02 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.0 abb62f0b9 -> 5869f8bf1


[SPARK-2283][SQL] Reset test environment before running PruningSuite

JIRA issue: [SPARK-2283](https://issues.apache.org/jira/browse/SPARK-2283)

If `PruningSuite` is run right after `HiveCompatibilitySuite`, the first test case fails because
`srcpart` table is cached in-memory by `HiveCompatibilitySuite`, but column pruning is not
implemented for `InMemoryColumnarTableScan` operator yet.

Author: Cheng Lian <lian.cs.zju@gmail.com>

Closes #1221 from liancheng/spark-2283 and squashes the following commits:

dc0b663 [Cheng Lian] SPARK-2283: reset test environment before running PruningSuite

(cherry picked from commit 7f196b009d26d4aed403b3c694f8b603601718e3)
Signed-off-by: Michael Armbrust <michael@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5869f8bf
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/5869f8bf
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/5869f8bf

Branch: refs/heads/branch-1.0
Commit: 5869f8bf13de39a098e4d386658b178bc1f12563
Parents: abb62f0
Author: Cheng Lian <lian.cs.zju@gmail.com>
Authored: Wed Jun 25 18:41:47 2014 -0700
Committer: Michael Armbrust <michael@databricks.com>
Committed: Wed Jun 25 18:41:58 2014 -0700

----------------------------------------------------------------------
 .../org/apache/spark/sql/hive/execution/PruningSuite.scala      | 5 +++++
 1 file changed, 5 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/5869f8bf/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/PruningSuite.scala
----------------------------------------------------------------------
diff --git a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/PruningSuite.scala
b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/PruningSuite.scala
index 3443444..4d7c84f 100644
--- a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/PruningSuite.scala
+++ b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/PruningSuite.scala
@@ -26,6 +26,11 @@ import scala.collection.JavaConversions._
  * A set of test cases that validate partition and column pruning.
  */
 class PruningSuite extends HiveComparisonTest {
+  // Column/partition pruning is not implemented for `InMemoryColumnarTableScan` yet, need
to reset
+  // the environment to ensure all referenced tables in this suites are not cached in-memory.
+  // Refer to https://issues.apache.org/jira/browse/SPARK-2283 for details.
+  TestHive.reset()
+
   // Column pruning tests
 
   createPruningTest("Column pruning - with partitioned table",


Mime
View raw message