Mailing-List: contact commits-help@singa.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@singa.incubator.apache.org
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: svn commit: r1692350 - in /incubator/singa/site/trunk/content:
 markdown/docs.md markdown/docs/programmer-guide.md
 markdown/docs/user-guide.md markdown/introduction.md
 markdown/quick-start.md site.xml
Date: Thu, 23 Jul 2015 06:03:42 -0000
To: commits@singa.incubator.apache.org
From: wangwei@apache.org
Message-Id: <20150723060343.9BEABAC0233@hades.apache.org>

Author: wangwei
Date: Thu Jul 23 06:03:42 2015
New Revision: 1692350

URL: http://svn.apache.org/r1692350
Log:
rename user-guide to programmer-guide

Added:
    incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md
Removed:
    incubator/singa/site/trunk/content/markdown/docs/user-guide.md
Modified:
    incubator/singa/site/trunk/content/markdown/docs.md
    incubator/singa/site/trunk/content/markdown/introduction.md
    incubator/singa/site/trunk/content/markdown/quick-start.md
    incubator/singa/site/trunk/content/site.xml

Modified: incubator/singa/site/trunk/content/markdown/docs.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs.md?rev=1692350&r1=1692349&r2=1692350&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs.md Thu Jul 23 06:03:42 2015
@@ -3,7 +3,8 @@
 ___
 
 * [Installation](docs/installation.html)
+* [Programmer Guide](docs/programmer-guide.html)
 * [System Architecture](docs/architecture.html)
 * [Communication](docs/communication.html)
-* [Neural Network Partition](docs/neuralnet-partition.html)
-* [Programming Model](docs/programming-model.html)
+* [Training Examples](docs/examples.html)
+

Added: incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md?rev=1692350&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md (added)
+++ incubator/singa/site/trunk/content/markdown/docs/programmer-guide.md Thu Jul 23 06:03:42 2015
@@ -0,0 +1,101 @@
+## Programming Model
+
+We describe the programming model of SINGA to provide users instructions of
+implementing a new model and submitting the training job. The programming model
+is made almost transparent to the underlying distributed environment. Hence
+users do not need to worry much about the communication and synchronization of
+nodes, which is discussed in [architecture](architecture.html) in details.
+
+### Deep learning training
+
+Deep learning is labeled as a feature learning technique, which usually
+consists of multiple layers.  Each layer is associated a feature transformation
+function. After going through all layers, the raw input feature (e.g., pixels
+of images) would be converted into a high-level feature that is easier for
+tasks like classification.
+
+Training a deep learning model is to find the optimal parameters involved in
+the transformation functions that generates good features for specific tasks.
+The goodness of a set of parameters is measured by a loss function, e.g.,
+[Cross-Entropy Loss](https://en.wikipedia.org/wiki/Cross_entropy). Since the
+loss functions are usually non-linear and non-convex, it is difficult to get a
+closed form solution. Normally, people uses the SGD algorithm which randomly
+initializes the parameters and then iteratively update them to reduce the loss.
+
+
+### Steps to submit a training job
+
+SINGA uses the stochastic gradient descent (SGD) algorithm to train parameters
+of deep learning models.  For each SGD iteration, there is a
+[Worker](architecture.html) computing gradients of parameters from the
+NeuralNet and a [Updater]() updating parameter values based on gradients. SINGA
+has implemented three algorithms for gradient calculation, namely Back
+propagation algorithm for feed-forward models, back-propagation through time
+for recurrent neural networks and contrastive divergence for energy models like
+RBM and DBM. Variant SGD updaters are also provided, including
+[AdaDelta](http://arxiv.org/pdf/1212.5701v1.pdf),
+[AdaGrad](http://www.magicbroom.info/Papers/DuchiHaSi10.pdf),
+[RMSProp](http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf),
+[Nesterov](http://scholar.google.com/citations?view_op=view_citation&amp;hl=en&amp;user=DJ8Ep8YAAAAJ&amp;citation_for_view=DJ8Ep8YAAAAJ:hkOj_22Ku90C).
+
+Consequently, what a user needs to do to submit a training job is
+
+  1. [Prepare the data](data.html) for training, validation and test.
+
+  2. [Implement the new Layers](layer.html) to support specific feature transformations
+  required in the new model.
+
+  3. Configure the training job including the [cluster setting](architecture.html)
+  and [model configuration](model-config.html)
+
+### Driver program
+
+Each training job has a driver program that
+
+  * registers the layers implemented by the user and,
+
+  * starts the [Trainer](https://github.com/apache/incubator-singa/blob/master/include/trainer/trainer.h)
+  by providing the job configuration.
+
+An example driver program is like
+
+    #include "singa.h"
+    #include "user-layer.h"  // header for user defined layers
+
+    DEFINE_int32(job, -1, "Job ID");  // job ID generated by the SINGA script
+    DEFINE_string(workspace, "examples/mnist/", "workspace of the training job");
+    DEFINE_bool(resume, false, "resume from checkpoint");
+
+    int main(int argc, char** argv) {
+      google::InitGoogleLogging(argv[0]);
+      gflags::ParseCommandLineFlags(&argc, &argv, true);
+
+      // register all user defined layers in user-layer.h
+      Register(kFooLayer, FooLayer);
+      ...
+
+      JobProto jobConf;
+      // read job configuration from text conf file
+      ReadProtoFromTextFile(&jobConf, FLAGS_workspace + "/job.conf");
+      Trainer trainer;
+      trainer.Start(FLAGS_job, jobConf, FLAGS_resume);
+    }
+
+Users can also configure the job in the driver program instead of writing the
+configuration file
+
+
+      JobProto jobConf;
+      jobConf.set_job_name("my singa job");
+      ... // configure cluster and model
+      Trainer trainer;
+      trainer.Start(FLAGS_job, jobConf, FLAGS_resume);
+
+We will provide helper functions to make the configuration easier in the
+future, like [keras](https://github.com/fchollet/keras).
+
+Compile and link the driver program with singa library to generate an
+executable, e.g., with name `mysinga`. To submit the job, just pass the path of
+the executable and the workspace to the singa job submission script
+
+    ./bin/singa-run.sh <path to mysinga> -workspace=<my job workspace>

Modified: incubator/singa/site/trunk/content/markdown/introduction.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/introduction.md?rev=1692350&r1=1692349&r2=1692350&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/introduction.md (original)
+++ incubator/singa/site/trunk/content/markdown/introduction.md Thu Jul 23 06:03:42 2015
@@ -59,10 +59,10 @@ general enough for programmers to overri
 
 ### Where to go from here
 
-  * SINGA [User guide](user-guide.html) describes how to submit a
+  * SINGA [Programmer guide](user-guide.html) describes how to submit a
   training job for your own deep learning model.
 
-  * SINGA [architecture](architecture.html) illustrates how different training frameworks are
+  * SINGA [Architecture](architecture.html) illustrates how different training frameworks are
    supported using a general system architecture.
 
   * [Training examples](examples.html) are provided to help users get started with SINGA.

Modified: incubator/singa/site/trunk/content/markdown/quick-start.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/quick-start.md?rev=1692350&r1=1692349&r2=1692350&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/quick-start.md (original)
+++ incubator/singa/site/trunk/content/markdown/quick-start.md Thu Jul 23 06:03:42 2015
@@ -129,7 +129,7 @@ there is no model partitioning. More det
 described in the [System Architecture](docs/architecture.html) page.
 -->
 
-#### Distributed Training
+### Distributed Training
 
 To train the model in a distributed environment, we first change the job
 configuration, e.g., using 2 worker groups (one worker per group) and 2 servers

Modified: incubator/singa/site/trunk/content/site.xml
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/site.xml?rev=1692350&r1=1692349&r2=1692350&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/site.xml (original)
+++ incubator/singa/site/trunk/content/site.xml Thu Jul 23 06:03:42 2015
@@ -55,7 +55,7 @@
 
     <menu name="Documentaion">
       <item name="Installation" href="docs/installation.html"/>
-      <item name="User Guide" href="docs/user-guide.html">
+      <item name="Programmer Guide" href="docs/programmer-guide.html">
         <item name ="Model Configuration" href="docs/model-config.html"/>
         <item name="Neural Network" href="docs/neuralnet.html"/>
         <item name="Layer" href="docs/layer.html"/>