spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sro...@apache.org
Subject spark git commit: [MINOR] [HIVE] Fix QueryPartitionSuite.
Date Sat, 02 May 2015 22:10:40 GMT
Repository: spark
Updated Branches:
  refs/heads/master bfcd528d6 -> 82c8c37c0


[MINOR] [HIVE] Fix QueryPartitionSuite.

At least in the version of Hive I tested on, the test was deleting
a temp directory generated by Hive instead of one containing partition
data. So fix the filter to only consider partition directories when
deciding what to delete.

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes #5854 from vanzin/hive-test-fix and squashes the following commits:

7594ae9 [Marcelo Vanzin] Fix typo.
729fa80 [Marcelo Vanzin] [minor] [hive] Fix QueryPartitionSuite.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/82c8c37c
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/82c8c37c
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/82c8c37c

Branch: refs/heads/master
Commit: 82c8c37c098e5886da65cea3108737744e270b91
Parents: bfcd528
Author: Marcelo Vanzin <vanzin@cloudera.com>
Authored: Sat May 2 23:10:35 2015 +0100
Committer: Sean Owen <sowen@cloudera.com>
Committed: Sat May 2 23:10:35 2015 +0100

----------------------------------------------------------------------
 .../scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/82c8c37c/sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala
----------------------------------------------------------------------
diff --git a/sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala b/sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala
index a787fa5..4990092 100644
--- a/sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala
+++ b/sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala
@@ -28,7 +28,7 @@ import org.apache.spark.util.Utils
 class QueryPartitionSuite extends QueryTest {
   import org.apache.spark.sql.hive.test.TestHive.implicits._
 
-  test("SPARK-5068: query data when path doesn't exists"){
+  test("SPARK-5068: query data when path doesn't exist"){
     val testData = TestHive.sparkContext.parallelize(
       (1 to 10).map(i => TestData(i, i.toString))).toDF()
     testData.registerTempTable("testData")
@@ -52,8 +52,9 @@ class QueryPartitionSuite extends QueryTest {
         ++ testData.toSchemaRDD.collect ++ testData.toSchemaRDD.collect)
 
     // delete the path of one partition
-    val folders = tmpDir.listFiles.filter(_.isDirectory)
-    Utils.deleteRecursively(folders(0))
+    tmpDir.listFiles
+      .find { f => f.isDirectory && f.getName().startsWith("ds=") }
+      .foreach { f => Utils.deleteRecursively(f) }
 
     // test for after delete the path
     checkAnswer(sql("select key,value from table_with_partition"),


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message