singa-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [4/5] incubator-singa git commit: SINGA-303 Create conda packages
Date Sat, 25 Feb 2017 05:27:45 GMT
SINGA-303 Create conda packages

Fix bugs:
_singa_wrap must be ended with .so (cannot use dylib) on osx
must link with libs from conda, there are two approaches
2. pass each dep lib path to cmake.
Otherwise the system lib (e.g., python) path would be linked into

@rpath/libpython2.7.dylib is replaced by a path RPATH list that has libpython2.7.dylib.
conda would store an entry @load_path/../../.. in RPATH for on osx.
@load_path is the path of

update readme


Branch: refs/heads/master
Commit: ca0be884c5cedc83fa10aab1547e9453c006a264
Parents: 493fc65
Author: xiezl <>
Authored: Fri Feb 24 10:22:56 2017 +0800
Committer: wangwei <>
Committed: Fri Feb 24 23:56:25 2017 +0800

 python/CMakeLists.txt  |  8 ++++++++
 tool/conda/    |  4 ++++
 tool/jenkins/ | 29 +++++++++++++++--------------
 3 files changed, 27 insertions(+), 14 deletions(-)
diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt
index 7445a57..177326e 100644
--- a/python/CMakeLists.txt
+++ b/python/CMakeLists.txt
@@ -129,3 +129,11 @@ CONFIGURE_FILE(${SETUP_PY_IN} ${SETUP_PY})
 #create python/singa/proto/
 FILE(WRITE ${CMAKE_BINARY_DIR}/python/singa/proto/ "")
+    change_suffix ALL
+    COMMAND ${CMAKE_COMMAND} -E rename "${CMAKE_BINARY_DIR}/python/singa/_singa_wrap.dylib"
+    COMMENT "change .dylib to .so in mac system"
+  )
+  ADD_DEPENDENCIES(change_suffix _singa_wrap)
diff --git a/tool/conda/ b/tool/conda/
index f3a84d9..926da71 100644
--- a/tool/conda/
+++ b/tool/conda/
@@ -15,8 +15,12 @@
 # limitations under the License.
+# to compile swig api files which depdend on numpy.i
 export export CPLUS_INCLUDE_PATH=`python -c "import numpy; print numpy.get_include()"`:$CPLUS_INCLUDE_PATH
+# to let cmake use the dependent libs installed by conda, including python
 mkdir build
 cd build
diff --git a/tool/jenkins/ b/tool/jenkins/
index 6a1cb1e..9a76ee8 100644
--- a/tool/jenkins/
+++ b/tool/jenkins/
@@ -1,12 +1,11 @@
 # Jenkins CI Support
 ## Introduction
-This documentation is to guide SINGA developers to setup Jenkins service.
+This documentation is to guide SINGA developers to setup Jenkins service to support continuous
integration on GPU systems. After each commit,
+1. SINGA should be compiled and tested automatically under different settings (e.g., OS and
+2. Convenient binaries should be generated automatically and archived.
-We use Jenkins to support continuous integration.
-After each commit, we want to automatically compile and test SINGA
-under different OS and settings.
-Those built binaries need to be archived for users to download.
+Continuous integration for CPU systems is enabled via [Travis](../travis).
 ## Install Jenkins
 [Jenkins Official Wiki](
@@ -56,18 +55,21 @@ Each node should configure the following environment variable
   * create conda package
     Execute shell - command -
-        git push --mirror <>
-        /root/miniconda/conda tool/conda/
+        git push -f
+        bash -ex tool/jenkins/ $lang
         export CONDA_BLD_PATH=/root/conda-bld-$BUILD_NUMBER
         mkdir $CONDA_BLD_PATH
-        /root/miniconda/anaconda -t $ANACONDA_UPLOAD_TOKEN upload -u $USER -l main $CONDA_BLD_PATH/$OS/singa-*.tar.bz2
+        /root/miniconda/bin/conda-build tool/conda
+        /root/miniconda/bin/anaconda -t ANACONDA_UPLOAD_TOKEN upload -u nusdbsystem -l main
$CONDA_BLD_PATH/linux-64/singa-*.tar.bz2 --force
     It first pushes to a mirror site to invoke travis-ci for CPU package creation;
-    Then it creates the conda package for GPU and uploads the package.
+    Then it compiles and runs unit tests;
+    Finally it creates the conda package for GPU and upload it.
 ### Post-build Actions
   * Publish JUnit test result report - Test report XMLs - ``**/gtest.xml, **/unittest.xml``
-  * (optional) Archive the artifacts - ``build/python/dist/**.whl, build/debian/**.deb``
+  * (optional) Archive the artifacts - ``build/debian/**.deb``
   * Send build artifacts (Debian package) over SSH for wheel
     * ../debian/ packages the .deb file into $BUILD_ID.tar.gz. Inside the tar file,
       the folder layout is `build_id/commit_hash/os_lang/*.deb`, where `os_lang` is the combination
of os version, device programming language (cuda/cpp/opencl) and cudnn version.
@@ -81,12 +83,11 @@ Each node should configure the following environment variable
     * The file links on the remote public server would be like
-            debian/32/84d56b7/ubuntu16.04-cpp/singa-1.0.1.deb
-            debian/32/84d56b7/ubuntu16.04-cuda8.0-cudnn5/singa-1.0.1.deb
+            debian/32/84d56b7/ubuntu14.04-cpp/singa-1.0.1.deb
+            debian/32/84d56b7/ubuntu14.04-cuda8.0-cudnn5/singa-1.0.1.deb
 ### Docker Images
-We provide in a number of singa docker [images](../docker) for Jenkins to use as slaves.
+We provide in a number of singa docker [images](./docker) for Jenkins to use as slaves.
 To run the docker images,
     nvidia-docker run --name <jenkins-slaveXX> -d <Image ID>

View raw message