spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From m...@apache.org
Subject git commit: [SPARK-2172] PySpark cannot import mllib modules in YARN-client mode
Date Thu, 26 Jun 2014 04:56:24 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.0 92b012502 -> fa167194c


[SPARK-2172] PySpark cannot import mllib modules in YARN-client mode

Include pyspark/mllib python sources as resources in the mllib.jar.
This way they will be included in the final assembly

Author: Szul, Piotr <Piotr.Szul@csiro.au>

Closes #1223 from piotrszul/branch-1.0 and squashes the following commits:

69d5174 [Szul, Piotr] Removed unsed resource directory src/main/resource from mllib pom
f8c52a0 [Szul, Piotr] [SPARK-2172] PySpark cannot import mllib modules in YARN-client mode
Include pyspark/mllib python sources as resources in the jar


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fa167194
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/fa167194
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/fa167194

Branch: refs/heads/branch-1.0
Commit: fa167194ce1b5898e4d7232346c9f86b2897a722
Parents: 92b0125
Author: Szul, Piotr <Piotr.Szul@csiro.au>
Authored: Wed Jun 25 21:55:49 2014 -0700
Committer: Xiangrui Meng <meng@databricks.com>
Committed: Wed Jun 25 21:55:49 2014 -0700

----------------------------------------------------------------------
 mllib/pom.xml | 8 ++++++++
 1 file changed, 8 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/fa167194/mllib/pom.xml
----------------------------------------------------------------------
diff --git a/mllib/pom.xml b/mllib/pom.xml
index 4c67de8..c6bc8bf 100644
--- a/mllib/pom.xml
+++ b/mllib/pom.xml
@@ -76,5 +76,13 @@
         <artifactId>scalatest-maven-plugin</artifactId>
       </plugin>
     </plugins>
+    <resources>
+      <resource>
+        <directory>../python</directory>
+        <includes>
+          <include>pyspark/mllib/*.py</include>
+        </includes>
+      </resource>
+    </resources>
   </build>
 </project>


Mime
View raw message