systemds-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [systemds] branch master updated: [SYSTEMDS-2524][DOC][1/2] Windows dev instructions
Date Tue, 14 Jul 2020 08:45:48 GMT
This is an automated email from the ASF dual-hosted git repository.

janardhan pushed a commit to branch master
in repository

The following commit(s) were added to refs/heads/master by this push:
     new 677037a  [SYSTEMDS-2524][DOC][1/2] Windows dev instructions
677037a is described below

commit 677037aae2ad7f546d1db9a427178f7ef15a2be5
Author: Janardhan Pulivarthi <>
AuthorDate: Tue Jul 14 14:13:53 2020 +0530

    [SYSTEMDS-2524][DOC][1/2] Windows dev instructions
      * Development setup for windows platform
      * Aims to provide an overview of the directory
    Closes #922.
 dev/docs/ | 87 +++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)

diff --git a/dev/docs/ b/dev/docs/
new file mode 100644
index 0000000..62e974d
--- /dev/null
+++ b/dev/docs/
@@ -0,0 +1,87 @@
+## Developing Apache SystemDS on Windows Platform
+These instructions will help you build Apache SystemDS from source code, which is the basis
for the engine
+and algorithms development. The following conventions will be used to refer to directories
on your machine:
+* `<USER_HOME>` is your home directory.
+* `<JDK_18_HOME>` is the root directory for the 1.8 JDK.
+* `<MAVEN_HOME>` is the root directory for the Apache Maven source code.
+* `<SYSTEMDS_HOME>` is the root directory for the SystemDS source code.
+* `<SPARK_HOME>` is the root directory for the Apache Spark source code.
+* `<HADOOP_HOME>` is the root directory for 
+* `<CUDA_HOME>`,`<CUDA_PATH>` is the top directory for NVIDIA GPU Computing Toolkit.
+Ex. For version `9.0`, it would like `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0`
+Also, make sure that `libnvvp`, `bin` folders are accessible to command line.
+*  `<CUDNN_PATH>` is the path 
+Ex. Corresponding to the CUDA version, `C:\Program Files\cudnn-9.0-windows10-x64-v7.2.1.38\cuda`
+#### Directory structure of the installation
+├── MAVEN_HOME = './maven'
+|   ├── bin
+|       ├── mvn.exe # and other executables
+├── SPARK_HOME = './spark'
+|   ├── bin
+|       ├── spark-shell # and other invocation commands
+├── HADOOP_HOME = './hadoop'
+│   ├── bin
+|       ├── wintuils.exe # The Default content layout and html file.
+### Getting Apache SystemDS Source Code
+SystemDS source code is available from [](
by either cloning or
+downloading a zip file (based on a branch) into `<SYSTEMDS_HOME>`. The default is the
*master* branch.
+git clone systemds
+The master branch contains the source code which will be used to create the next major version
of Apache SystemDS.
+_**Speed Tip:**_ If the complete repository history isn't needed then using a shallow clone
(`git clone --depth 1`) will
+save significant time.
+### Building SystemDS source code
+`IntelliJ IDEA` or `Eclipse` is preferred for best developer experience.
+#### Opening the IntelliJ Source Code for Build
+Using IntelliJ IDEA **File | Open**, select the `<SYSTEMDS_HOME>` directory. 
+* If IntelliJ IDEA displays an error about a missing or out of date required plugin (e.g.
+  [enable, upgrade, or install that plugin](
and restart IntelliJ IDEA.
+#### IntelliJ Build Configuration
+JDK version 1.8 (u151 or newer) is required for building and developing for SystemDS developement.
+1. Using IntelliJ IDEA, [configure]( a JDK named
"**1.8**", pointing to `<JDK_18_HOME>`.
+   * If not already present, add `<JDK_18_HOME>/lib/tools.jar` [to the Classpath](
+     for the **1.8** JDK.
+2. If the _Maven Integration_ plugin is disabled, [add the path variable](
+   "**MAVEN_REPOSITORY**" pointing to `<USER_HOME>/.m2/repository` directory.
+3. _**Speed Tip:**_ If you have enough RAM on your computer,
+   [configure the compiler settings](
+   to enable the "Compile independent modules in parallel" option. Also set the "User-local
build process VM options" to `-Xmx2G`.
+   These changes will greatly reduce the compile time.
+4. Now, selecting the IntelliJ IDEA **Build | Build module 'systemds'** option starts the
maven build.
+5. _**Speed Tip:**_ If the development machine have enough RAM,
+   [configure the compiler settings](
+   to enable the "Compile independent modules in parallel" option.
+#### Building the Source Code
+To build SystemDS from source, choose **Build | Build Project** from the main menu.
+To maven build, run the `mvn clean package` command in `<SYSTEMDS_HOME>` directory.
See the `pom.xml` file for details.
+### Testing
+To run the SystemDS built from source, choose **Run | Run** from the main menu.
+To run tests on the build, apply these setting to the **Run | Edit Configurations... | Defaults
| JUnit** configuration tab:
+  * Working dir: `<SYSTEMDS_HOME>`
+  * VM options:
+    * `-ea`

View raw message