spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pwend...@apache.org
Subject git commit: [SPARK-1511] use Files.move instead of renameTo in TestUtils.scala
Date Wed, 16 Apr 2014 21:56:32 GMT
Repository: spark
Updated Branches:
  refs/heads/master 725925cf2 -> 10b1c59dc


[SPARK-1511] use Files.move instead of renameTo in TestUtils.scala

JIRA issue:[SPARK-1511](https://issues.apache.org/jira/browse/SPARK-1511)

TestUtils.createCompiledClass method use renameTo() to move files which fails when the src
and dest files are in different disks or partitions. This pr uses Files.move() instead. The
move method will try to use renameTo() and then fall back to copy() and delete(). I think
this should handle this issue.

I didn't found a test suite for this file, so I add file existence detection after file moving.

Author: Ye Xianjin <advancedxy@gmail.com>

Closes #427 from advancedxy/SPARK-1511 and squashes the following commits:

a2b97c7 [Ye Xianjin] Based on @srowen's comment, assert file existence.
6f95550 [Ye Xianjin] use Files.move instead of renameTo to handle the src and dest files are
in different disks or partitions.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/10b1c59d
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/10b1c59d
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/10b1c59d

Branch: refs/heads/master
Commit: 10b1c59dcc9ca2c1dafa02cb3ea298f3b33fc914
Parents: 725925c
Author: Ye Xianjin <advancedxy@gmail.com>
Authored: Wed Apr 16 14:56:22 2014 -0700
Committer: Patrick Wendell <pwendell@gmail.com>
Committed: Wed Apr 16 14:56:22 2014 -0700

----------------------------------------------------------------------
 core/src/main/scala/org/apache/spark/TestUtils.scala | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/10b1c59d/core/src/main/scala/org/apache/spark/TestUtils.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/TestUtils.scala b/core/src/main/scala/org/apache/spark/TestUtils.scala
index f3f59e4..8ae0215 100644
--- a/core/src/main/scala/org/apache/spark/TestUtils.scala
+++ b/core/src/main/scala/org/apache/spark/TestUtils.scala
@@ -100,9 +100,14 @@ private[spark] object TestUtils {
 
     val fileName = className + ".class"
     val result = new File(fileName)
-    if (!result.exists()) throw new Exception("Compiled file not found: " + fileName)
+    assert(result.exists(), "Compiled file not found: " + result.getAbsolutePath())
     val out = new File(destDir, fileName)
-    result.renameTo(out)
+
+    // renameTo cannot handle in and out files in different filesystems
+    // use google's Files.move instead
+    Files.move(result, out)
+
+    assert(out.exists(), "Destination file not moved: " + out.getAbsolutePath())
     out
   }
 }


Mime
View raw message