spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From andrewo...@apache.org
Subject git commit: [SPARK-3061] Fix Maven build under Windows
Date Tue, 09 Sep 2014 22:26:34 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.1 e5f77ae97 -> 23fd3e8b9


[SPARK-3061] Fix Maven build under Windows

The Maven build was failing on Windows because it tried to call the unix `unzip` utility to
extract the Py4J files into core's build directory.  I've fixed this issue by using the `maven-antrun-plugin`
to perform the unzipping.

I also fixed an issue that prevented tests from running under Windows:

In the Maven ScalaTest plugin, the filename listed in <filereports> is placed under
the <reportsDirectory>; the current code places it in a subdirectory of reportsDirectory,
e.g.

```
${project.build.directory}/surefire-reports/${project.build.directory}/SparkTestSuite.txt
```

This caused problems under Windows because it would try to create a subdirectory named "c:\\".

Note that the tests still fail under Windows (for other reasons); this PR just allows them
to run and fail rather than crash when trying to create the test reports directory.

Author: Josh Rosen <joshrosen@apache.org>
Author: Josh Rosen <rosenville@gmail.com>
Author: Josh Rosen <joshrosen@databricks.com>

Closes #2165 from JoshRosen/windows-support and squashes the following commits:

651d210 [Josh Rosen] Unzip to python/build instead of core/build
fbf3e61 [Josh Rosen] 4 spaces -> 2 spaces
e347668 [Josh Rosen] Fix Maven scalatest filereports path:
4994af1 [Josh Rosen] [SPARK-3061] Use maven-antrun-plugin to unzip Py4J.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/23fd3e8b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/23fd3e8b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/23fd3e8b

Branch: refs/heads/branch-1.1
Commit: 23fd3e8b95845b956b3c90df660bc3cf0ed42d28
Parents: e5f77ae
Author: Josh Rosen <joshrosen@apache.org>
Authored: Tue Sep 2 10:45:14 2014 -0700
Committer: Andrew Or <andrewor14@gmail.com>
Committed: Tue Sep 9 15:26:22 2014 -0700

----------------------------------------------------------------------
 core/pom.xml | 18 ++++++------------
 pom.xml      |  2 +-
 2 files changed, 7 insertions(+), 13 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/23fd3e8b/core/pom.xml
----------------------------------------------------------------------
diff --git a/core/pom.xml b/core/pom.xml
index debc4dd..0b9486f 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -300,26 +300,20 @@
       </plugin>
       <!-- Unzip py4j so we can include its files in the jar -->
       <plugin>
-        <groupId>org.codehaus.mojo</groupId>
-        <artifactId>exec-maven-plugin</artifactId>
-        <version>1.2.1</version>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-antrun-plugin</artifactId>
         <executions>
           <execution>
             <phase>generate-resources</phase>
             <goals>
-              <goal>exec</goal>
+              <goal>run</goal>
             </goals>
           </execution>
         </executions>
         <configuration>
-          <executable>unzip</executable>
-          <workingDirectory>../python</workingDirectory>
-          <arguments>
-            <argument>-o</argument>
-            <argument>lib/py4j*.zip</argument>
-            <argument>-d</argument>
-            <argument>build</argument>
-          </arguments>
+          <tasks>
+            <unzip src="../python/lib/py4j-0.8.2.1-src.zip" dest="../python/build" />
+          </tasks>
         </configuration>
       </plugin>
     </plugins>

http://git-wip-us.apache.org/repos/asf/spark/blob/23fd3e8b/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 2a2e1e3..66458e2 100644
--- a/pom.xml
+++ b/pom.xml
@@ -878,7 +878,7 @@
           <configuration>
             <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
             <junitxml>.</junitxml>
-            <filereports>${project.build.directory}/SparkTestSuite.txt</filereports>
+            <filereports>SparkTestSuite.txt</filereports>
             <argLine>-Xmx3g -XX:MaxPermSize=${MaxPermGen} -XX:ReservedCodeCacheSize=512m</argLine>
             <stderr />
             <systemProperties>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message