singa-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From build...@apache.org
Subject svn commit: r965910 [2/3] - in /websites/staging/singa/trunk/content: ./ community/ develop/ docs/
Date Fri, 18 Sep 2015 15:11:55 GMT
Modified: websites/staging/singa/trunk/content/docs/layer.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/layer.html (original)
+++ websites/staging/singa/trunk/content/docs/layer.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; Layers</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           
@@ -483,13 +489,14 @@
         <div id="bodyColumn"  class="span10" >
                                   
             <h1>Layers</h1>
+<hr />
 <p>Layer is a core abstraction in SINGA. It performs a variety of feature transformations for extracting high-level features, e.g., loading raw features, parsing RGB values, doing convolution transformation, etc.</p>
 <p>The <i>Basic user guide</i> section introduces the configuration of a built-in layer. <i>Advanced user guide</i> explains how to extend the base Layer class to implement users&#x2019; functions.</p>
 <div class="section">
 <h2><a name="Basic_user_guide"></a>Basic user guide</h2>
 <div class="section">
 <h3><a name="Layer_configuration"></a>Layer configuration</h3>
-<p>The configurations of three layers from the <a class="externalLink" href="http://singa.incubator.apache.org/docs/mlp">MLP example</a> is shown below,</p>
+<p>The configurations of three layers from the <a href="mlp.html">MLP example</a> is shown below,</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">layer {
@@ -522,13 +529,13 @@ layer{
   
 <li><tt>type</tt>: an integer used for identifying a Layer subclass. The types of built-in  layers are listed in LayerType (defined in job.proto).  For user-defined layer subclasses, <tt>user_type</tt> of string should be used instead of <tt>type</tt>.  The detail is explained in the <a href="#newlayer">last section</a> of this page.</li>
   
-<li><tt>srclayers</tt>: one or more layer names, for identifying the source layers.  In SINGA, all connections are <a class="externalLink" href="http://singa.incubator.apache.org/docs/neural-net">converted</a> to directed connections.</li>
+<li><tt>srclayers</tt>: one or more layer names, for identifying the source layers.  In SINGA, all connections are <a href="neural-net.html">converted</a> to directed connections.</li>
   
-<li><tt>exclude</tt>: an enumerate value of type <a href="">Phase</a>, can be {kTest, kValidation,  kTrain}. It is used to filter this layer when creating the  <a class="externalLink" href="http://singa.incubator.apache.org/docs/neural-net">NeuralNet</a> for the excluding phase. E.g.,  the &#x201c;data&#x201d; layer would be filtered when creating the NeuralNet instance for test phase.</li>
+<li><tt>exclude</tt>: an enumerate value of type <a href="">Phase</a>, can be {kTest, kValidation,  kTrain}. It is used to filter this layer when creating the  <a href="neural-net.html">NeuralNet</a> for the excluding phase. E.g.,  the &#x201c;data&#x201d; layer would be filtered when creating the NeuralNet instance for test phase.</li>
   
-<li><tt>param</tt>: configuration for a <a class="externalLink" href="http://singa.incubator.apache.org/docs/param">Param</a> instance.  There can be multiple Param objects in one layer.</li>
+<li><tt>param</tt>: configuration for a <a href="param.html">Param</a> instance.  There can be multiple Param objects in one layer.</li>
   
-<li><tt>partition_dim</tt>: integer value indicating the partition dimension of this  layer. -1 (the default value) for no partitioning, 0 for partitioning on batch dimension, 1 for  partitioning on feature dimension. It is used by  <a class="externalLink" href="http://singa.incubator.apache.org/docs/neural-net">CreateGraph</a> for partitioning the neural net.</li>
+<li><tt>partition_dim</tt>: integer value indicating the partition dimension of this  layer. -1 (the default value) for no partitioning, 0 for partitioning on batch dimension, 1 for  partitioning on feature dimension. It is used by  <a href="neural-net.html">CreateGraph</a> for partitioning the neural net.</li>
 </ul>
 <p>Different layers may have different configurations. These configurations are defined in <tt>&lt;type&gt;_conf</tt>. E.g., the &#x201c;data&#x201d; layer has <tt>sharddata_conf</tt> and &#x201c;fc1&#x201d; layer has <tt>innerproduct_conf</tt>. The subsequent sections explain the functionality of each built-in layer and how to configure it,</p></div>
 <div class="section">
@@ -539,9 +546,9 @@ layer{
   
 <li>Data layers for loading records (e.g., images) from [disk], HDFS or network into memory.</li>
   
-<li>Parser layers for parsing features, labels, etc. from records, into <a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1Blob.html">Blob</a>.</li>
+<li>Parser layers for parsing features, labels, etc. from records, into <a href="../api/classsinga_1_1Blob.html">Blob</a>.</li>
   
-<li>Neuron layers for feature transformation, e.g., <a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1ConvolutionLayer.html">convolution</a>, <a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1PoolingLayer.html">pooling</a>, dropout, etc.</li>
+<li>Neuron layers for feature transformation, e.g., <a href="../api/classsinga_1_1ConvolutionLayer.html">convolution</a>, <a href="../api/classsinga_1_1PoolingLayer.html">pooling</a>, dropout, etc.</li>
   
 <li>Loss layers for measuring the training objective loss, e.g., [cross entropy-loss] or [Euclidean loss].</li>
   
@@ -550,11 +557,14 @@ layer{
 <li>Connection layers for connecting layers when the neural net is partitioned.</li>
 </ul>
 <div class="section">
-<h4><a name="Data_Layers"></a>Data Layers</h4>
-<p>Data layers load training/testing data and convert them into <a class="externalLink" href="http://singa.incubator.apache.org/docs/data">Record</a>s, which are parsed by parser layers. The data source can be disk file, HDFS, database or network.</p>
+<h4><a name="Input_layers"></a>Input layers</h4>
+<p>Input layers load training/test data from disk or other places (e.g., HDFS or network) into memory.</p>
+<div class="section">
+<h5><a name="DataLayer"></a>DataLayer</h5>
+<p>DataLayer loads training/testing data as <a class="externalLink" href="http://singa.incubator.apache.org/docs/data">Record</a>s, which are parsed by parser layers.</p></div>
 <div class="section">
 <h5><a name="ShardDataLayer"></a>ShardDataLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1ShardDataLayer.html">ShardDataLayer</a> is used to read data from disk file. The file should be created using <a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1DataShard.html">DataShard</a> class. With the data file prepared, users configure the layer as</p>
+<p><a href="../api/classsinga_1_1ShardDataLayer.html">ShardDataLayer</a> is a subclass of DataLayer, which reads Records from disk file. The file should be created using <a href="../api/classsinga_1_1DataShard.html">DataShard</a> class. With the data file prepared, users configure the layer as</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kShardData
@@ -576,20 +586,20 @@ lmdbdata_conf {
   batchsize: int
   random_skip: int
 }
-</pre></div></div></div></div>
+</pre></div></div></div>
 <div class="section">
-<h4><a name="Parser_Layers"></a>Parser Layers</h4>
-<p>Parser layers get a vector of Records from data layers and parse features into a Blob.</p>
+<h5><a name="ParserLayer"></a>ParserLayer</h5>
+<p>It get a vector of Records from DataLayer and parse features into a Blob.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">virtual void ParseRecords(Phase phase, const vector&lt;Record&gt;&amp; records, Blob&lt;float&gt;* blob) = 0;
-</pre></div></div>
+</pre></div></div></div>
 <div class="section">
 <h5><a name="LabelLayer"></a>LabelLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1LabelLayer.html">LabelLayer</a> is used to parse a single label from each Record. Consequently, it will put $b$ (mini-batch size) values into the Blob. It has no specific configuration fields.</p></div>
+<p><a href="../api/classsinga_1_1LabelLayer.html">LabelLayer</a> is a subclass of ParserLayer. It parses a single label from each Record. Consequently, it will put $b$ (mini-batch size) values into the Blob. It has no specific configuration fields.</p></div>
 <div class="section">
 <h5><a name="MnistImageLayer"></a>MnistImageLayer</h5>
-<p>[MnistImageLayer] parses the pixel values of each image from the MNIST dataset. The pixel values may be normalized as <tt>x/norm_a - norm_b</tt>. For example, if <tt>norm_a</tt> is set to 255 and <tt>norm_b</tt> is set to 0, then every pixel will be normalized into [0, 1].</p>
+<p>[MnistImageLayer] is a subclass of ParserLayer. It parses the pixel values of each image from the MNIST dataset. The pixel values may be normalized as <tt>x/norm_a - norm_b</tt>. For example, if <tt>norm_a</tt> is set to 255 and <tt>norm_b</tt> is set to 0, then every pixel will be normalized into [0, 1].</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kMnistImage
@@ -600,7 +610,7 @@ mnistimage_conf {
 </pre></div></div></div>
 <div class="section">
 <h5><a name="RGBImageLayer"></a>RGBImageLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1RGBImageLayer.html">RGBImageLayer</a> parses the RGB values of one image from each Record. It may also apply some transformations, e.g., cropping, mirroring operations. If the <tt>meanfile</tt> is specified, it should point to a path that contains one Record for the mean of each pixel over all training images.</p>
+<p><a href="../api/classsinga_1_1RGBImageLayer.html">RGBImageLayer</a> is a subclass of ParserLayer. It parses the RGB values of one image from each Record. It may also apply some transformations, e.g., cropping, mirroring operations. If the <tt>meanfile</tt> is specified, it should point to a path that contains one Record for the mean of each pixel over all training images.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kRGBImage
@@ -613,7 +623,7 @@ rgbimage_conf {
 </pre></div></div></div></div>
 <div class="section">
 <h4><a name="PrefetchLayer"></a>PrefetchLayer</h4>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1PrefetchLayer.html">PrefetchLayer</a> embeds data layers and parser layers to do data prefeching. It will launch a thread to call the data layers and parser layers to load and extract features. It ensures that the I/O task and computation task can work simultaneously. One example PrefetchLayer configuration is,</p>
+<p><a href="../api/classsinga_1_1PrefetchLayer.html">PrefetchLayer</a> embeds other input layers to do data prefeching. It will launch a thread to call the embedded layers to load and extract features. It ensures that the I/O task and computation task can work simultaneously. One example PrefetchLayer configuration is,</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">layer {
@@ -644,7 +654,7 @@ rgbimage_conf {
 <p>Neuron layers conduct feature transformations.</p>
 <div class="section">
 <h5><a name="ConvolutionLayer"></a>ConvolutionLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1ConvolutionLayer.html">ConvolutionLayer</a> conducts convolution transformation.</p>
+<p><a href="../api/classsinga_1_1ConvolutionLayer.html">ConvolutionLayer</a> conducts convolution transformation.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kConvolution
@@ -660,7 +670,7 @@ param { } # bias vector
 <p>The int value <tt>num_filters</tt> stands for the count of the applied filters; the int value <tt>kernel</tt> stands for the convolution kernel size (equal width and height); the int value <tt>stride</tt> stands for the distance between the successive filters; the int value <tt>pad</tt> pads each with a given int number of pixels border of zeros.</p></div>
 <div class="section">
 <h5><a name="InnerProductLayer"></a>InnerProductLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1InnerProductLayer.html">InnerProductLayer</a> is fully connected with its (single) source layer. Typically, it has two parameter fields, one for weight matrix, and the other for bias vector. It rotates the feature of the source layer (by multiplying with weight matrix) and shifts it (by adding the bias vector).</p>
+<p><a href="../api/classsinga_1_1InnerProductLayer.html">InnerProductLayer</a> is fully connected with its (single) source layer. Typically, it has two parameter fields, one for weight matrix, and the other for bias vector. It rotates the feature of the source layer (by multiplying with weight matrix) and shifts it (by adding the bias vector).</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kInnerProduct
@@ -672,7 +682,7 @@ param { } # bias vector
 </pre></div></div></div>
 <div class="section">
 <h5><a name="PoolingLayer"></a>PoolingLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1PoolingLayer.html">PoolingLayer</a> is used to do a normalization (or averaging or sampling) of the feature vectors from the source layer.</p>
+<p><a href="../api/classsinga_1_1PoolingLayer.html">PoolingLayer</a> is used to do a normalization (or averaging or sampling) of the feature vectors from the source layer.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kPooling
@@ -693,16 +703,16 @@ pooling_conf {
 </ul></div>
 <div class="section">
 <h5><a name="ReLULayer"></a>ReLULayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1ReLULayer.html">ReLuLayer</a> has rectified linear neurons, which conducts the following transformation, <tt>f(x) = Max(0, x)</tt>. It has no specific configuration fields.</p></div>
+<p><a href="../api/classsinga_1_1ReLULayer.html">ReLuLayer</a> has rectified linear neurons, which conducts the following transformation, <tt>f(x) = Max(0, x)</tt>. It has no specific configuration fields.</p></div>
 <div class="section">
 <h5><a name="TanhLayer"></a>TanhLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1TanhLayer.html">TanhLayer</a> uses the tanh as activation function, i.e., <tt>f(x)=tanh(x)</tt>. It has no specific configuration fields.</p></div>
+<p><a href="../api/classsinga_1_1TanhLayer.html">TanhLayer</a> uses the tanh as activation function, i.e., <tt>f(x)=tanh(x)</tt>. It has no specific configuration fields.</p></div>
 <div class="section">
 <h5><a name="SigmoidLayer"></a>SigmoidLayer</h5>
 <p>[SigmoidLayer] uses the sigmoid (or logistic) as activation function, i.e., <tt>f(x)=sigmoid(x)</tt>. It has no specific configuration fields.</p></div>
 <div class="section">
 <h5><a name="Dropout_Layer"></a>Dropout Layer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/asssinga_1_1DropoutLayer.html">DropoutLayer</a> is a layer that randomly dropouts some inputs. This scheme helps deep learning model away from over-fitting.</p>
+<p><a href="../api/asssinga_1_1DropoutLayer.html">DropoutLayer</a> is a layer that randomly dropouts some inputs. This scheme helps deep learning model away from over-fitting.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kDropout
@@ -712,7 +722,7 @@ dropout_conf {
 </pre></div></div></div>
 <div class="section">
 <h5><a name="LRNLayer"></a>LRNLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1LRNLayer.html">LRNLayer</a>, (Local Response Normalization), normalizes over the channels.</p>
+<p><a href="../api/classsinga_1_1LRNLayer.html">LRNLayer</a>, (Local Response Normalization), normalizes over the channels.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kLRN
@@ -728,7 +738,7 @@ lrn_conf {
 <p>Loss layers measures the objective training loss.</p>
 <div class="section">
 <h5><a name="SoftmaxLossLayer"></a>SoftmaxLossLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1SoftmaxLossLayer.html">SoftmaxLossLayer</a> is a combination of the Softmax transformation and Cross-Entropy loss. It applies Softmax firstly to get a prediction probability for each output unit (neuron) and compute the cross-entropy against the ground truth. It is generally used as the final layer to generate labels for classification tasks.</p>
+<p><a href="../api/classsinga_1_1SoftmaxLossLayer.html">SoftmaxLossLayer</a> is a combination of the Softmax transformation and Cross-Entropy loss. It applies Softmax firstly to get a prediction probability for each output unit (neuron) and compute the cross-entropy against the ground truth. It is generally used as the final layer to generate labels for classification tasks.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kSoftmaxLoss
@@ -738,10 +748,11 @@ softmaxloss_conf {
 </pre></div></div>
 <p>The configuration field <tt>topk</tt> is for selecting the labels with <tt>topk</tt> probabilities as the prediction results. It is tedious for users to view the prediction probability of every label.</p></div></div>
 <div class="section">
-<h4><a name="Other_Layers"></a>Other Layers</h4>
+<h4><a name="ConnectionLayer"></a>ConnectionLayer</h4>
+<p>Subclasses of ConnectionLayer are utility layers that connects other layers due to neural net partitioning or other cases.</p>
 <div class="section">
 <h5><a name="ConcateLayer"></a>ConcateLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1ConcateLayer.html">ConcateLayer</a> connects more than one source layers to concatenate their feature blob along given dimension.</p>
+<p><a href="../api/classsinga_1_1ConcateLayer.html">ConcateLayer</a> connects more than one source layers to concatenate their feature blob along given dimension.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kConcate
@@ -751,7 +762,7 @@ concate_conf {
 </pre></div></div></div>
 <div class="section">
 <h5><a name="SliceLayer"></a>SliceLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1SliceLayer.html">SliceLayer</a> connects to more than one destination layers to slice its feature blob along given dimension.</p>
+<p><a href="../api/classsinga_1_1SliceLayer.html">SliceLayer</a> connects to more than one destination layers to slice its feature blob along given dimension.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kSlice
@@ -761,7 +772,7 @@ slice_conf {
 </pre></div></div></div>
 <div class="section">
 <h5><a name="SplitLayer"></a>SplitLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1SplitLayer.html">SplitLayer</a> connects to more than one destination layers to replicate its feature blob.</p>
+<p><a href="../api/classsinga_1_1SplitLayer.html">SplitLayer</a> connects to more than one destination layers to replicate its feature blob.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">type: kSplit
@@ -771,7 +782,10 @@ split_conf {
 </pre></div></div></div>
 <div class="section">
 <h5><a name="BridgeSrcLayer__BridgeDstLayer"></a>BridgeSrcLayer &amp; BridgeDstLayer</h5>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1BridgeSrcLayer.html">BridgeSrcLayer</a> &amp; <a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1BridgeDstLayer.html">BridgeDstLayer</a> are utility layers assisting data (e.g., feature or gradient) transferring due to neural net partitioning. These two layers are added implicitly. Users typically do not need to configure them in their neural net configuration.</p></div></div></div></div>
+<p><a href="../api/classsinga_1_1BridgeSrcLayer.html">BridgeSrcLayer</a> &amp; <a href="../api/classsinga_1_1BridgeDstLayer.html">BridgeDstLayer</a> are utility layers assisting data (e.g., feature or gradient) transferring due to neural net partitioning. These two layers are added implicitly. Users typically do not need to configure them in their neural net configuration.</p></div></div></div>
+<div class="section">
+<h3><a name="OutputLayer"></a>OutputLayer</h3>
+<p>It write the prediction results or the extracted features into file, HTTP stream or other places. Currently SINGA has not implemented any specific output layer.</p></div></div>
 <div class="section">
 <h2><a name="Advanced_user_guide"></a>Advanced user guide</h2>
 <p>The base Layer class is introduced in this section, followed by how to implement a new Layer subclass.</p>
@@ -785,8 +799,8 @@ split_conf {
 Blob&lt;float&gt; data_, grad_;
 vector&lt;Layer*&gt; srclayers_, dstlayers_;
 </pre></div></div>
-<p>The base layer class keeps the user configuration in <tt>layer_proto_</tt>. Source layers and destination layers are stored in <tt>srclayers_</tt> and <tt>dstlayers_</tt>, respectively. Almost all layers has $b$ (mini-batch size) feature vectors, which are stored in the <tt>data_</tt> <a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1Blob.html">Blob</a> (A Blob is a chunk of memory space, proposed in <a class="externalLink" href="http://caffe.berkeleyvision.org/">Caffe</a>). There are layers without feature vectors; instead, they use other layers&#x2019; feature vectors. In this case, the <tt>data_</tt> field is not used. The <tt>grad_</tt> Blob is for storing the gradients of the objective loss w.r.t. the <tt>data_</tt> Blob. It is necessary in <a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1BPWorker.html">BP algorithm</a>, hence we put it as a member of the base class. For <a class="externalLink" href="http://sin
 ga.incubator.apache.org/api/classsinga_1_1CDWorker.html">CD algorithm</a>, the <tt>grad_</tt> field is not used; instead, the layer from RBM may have a Blob for the positive phase feature and a Blob for the negative phase feature. For a recurrent layer in RNN, the feature blob contains one vector per internal layer.</p>
-<p>If a layer has parameters, these parameters are declared using type <a class="externalLink" href="http://singa.incubator.apache.org/docs/param">Param</a>. Since some layers do not have parameters, we do not declare any <tt>Param</tt> in the base layer class.</p></div>
+<p>The base layer class keeps the user configuration in <tt>layer_proto_</tt>. Source layers and destination layers are stored in <tt>srclayers_</tt> and <tt>dstlayers_</tt>, respectively. Almost all layers has $b$ (mini-batch size) feature vectors, which are stored in the <tt>data_</tt> <a href="../api/classsinga_1_1Blob.html">Blob</a> (A Blob is a chunk of memory space, proposed in <a class="externalLink" href="http://caffe.berkeleyvision.org/">Caffe</a>). There are layers without feature vectors; instead, they use other layers&#x2019; feature vectors. In this case, the <tt>data_</tt> field is not used. The <tt>grad_</tt> Blob is for storing the gradients of the objective loss w.r.t. the <tt>data_</tt> Blob. It is necessary in <a href="../api/classsinga_1_1BPWorker.html">BP algorithm</a>, hence we put it as a member of the base class. For <a href="../api/classsinga_1_1CDWorker.html">CD algorithm</a>, the <tt>grad_</tt> field is not used; instead, the layer from RBM may have a Blob
  for the positive phase feature and a Blob for the negative phase feature. For a recurrent layer in RNN, the feature blob contains one vector per internal layer.</p>
+<p>If a layer has parameters, these parameters are declared using type <a href="param.html">Param</a>. Since some layers do not have parameters, we do not declare any <tt>Param</tt> in the base layer class.</p></div>
 <div class="section">
 <h4><a name="Functions"></a>Functions</h4>
 
@@ -795,11 +809,11 @@ vector&lt;Layer*&gt; srclayers_, dstlaye
 virtual void ComputeFeature(Phase phase, Metric* perf) = 0;
 virtual void ComputeGradient(Phase phase) = 0;
 </pre></div></div>
-<p>The <tt>Setup</tt> function reads user configuration, i.e. <tt>proto</tt>, and information from source layers, e.g., mini-batch size, to set the shape of the <tt>data_</tt> (and <tt>grad_</tt>) field as well as some other layer specific fields. If <tt>npartitions</tt> is larger than 1, then users need to reduce the sizes of <tt>data_</tt>, <tt>grad_</tt> Blobs or Param objects. For example, if the <tt>partition_dim=0</tt> and there is no source layer, e.g., this layer is a (bottom) data layer, then its <tt>data_</tt> and <tt>grad_</tt> Blob should have <tt>b/npartitions</tt> feature vectors; If the source layer is also partitioned on dimension 0, then this layer should have the same number of feature vectors as the source layer. More complex partition cases are discussed in <a class="externalLink" href="http://singa.incubator.apache.org/docs/neural-net/#neural-net-partitioning">Neural net partitioning</a>. Typically, the Setup function just set the shapes of <tt>data_</tt> Blobs 
 and Param objects. Memory will not be allocated until computation over the data structure happens.</p>
-<p>The <tt>ComputeFeature</tt> function evaluates the feature blob by transforming (e.g. convolution and pooling) features from the source layers. <tt>ComputeGradient</tt> computes the gradients of parameters associated with this layer. These two functions are invoked by the <a class="externalLink" href="http://singa.incubator.apache.org/docs/train-one-batch">TrainOneBatch</a> function during training. Hence, they should be consistent with the <tt>TrainOneBatch</tt> function. Particularly, for feed-forward and RNN models, they are trained using <a class="externalLink" href="http://singa.incubator.apache.org/docs/train-one-batch/#back-propagation">BP algorithm</a>, which requires each layer&#x2019;s <tt>ComputeFeature</tt> function to compute <tt>data_</tt> based on source layers, and requires each layer&#x2019;s <tt>ComputeGradient</tt> to compute gradients of parameters and source layers&#x2019; <tt>grad_</tt>. For energy models, e.g., RBM, they are trained by <a class="externalLin
 k" href="http://singa.incubator.apache.org/docs/train-one-batch/#contrastive-divergence">CD algorithm</a>, which requires each layer&#x2019;s <tt>ComputeFeature</tt> function to compute the feature vectors for the positive phase or negative phase depending on the <tt>phase</tt> argument, and requires the <tt>ComputeGradient</tt> function to only compute parameter gradients. For some layers, e.g., loss layer or output layer, they can put the loss or prediction result into the <tt>metric</tt> argument, which will be averaged and displayed periodically.</p></div></div>
+<p>The <tt>Setup</tt> function reads user configuration, i.e. <tt>proto</tt>, and information from source layers, e.g., mini-batch size, to set the shape of the <tt>data_</tt> (and <tt>grad_</tt>) field as well as some other layer specific fields. If <tt>npartitions</tt> is larger than 1, then users need to reduce the sizes of <tt>data_</tt>, <tt>grad_</tt> Blobs or Param objects. For example, if the <tt>partition_dim=0</tt> and there is no source layer, e.g., this layer is a (bottom) data layer, then its <tt>data_</tt> and <tt>grad_</tt> Blob should have <tt>b/npartitions</tt> feature vectors; If the source layer is also partitioned on dimension 0, then this layer should have the same number of feature vectors as the source layer. More complex partition cases are discussed in <a href="neural-net.html#neural-net-partitioning">Neural net partitioning</a>. Typically, the Setup function just set the shapes of <tt>data_</tt> Blobs and Param objects. Memory will not be allocated until co
 mputation over the data structure happens.</p>
+<p>The <tt>ComputeFeature</tt> function evaluates the feature blob by transforming (e.g. convolution and pooling) features from the source layers. <tt>ComputeGradient</tt> computes the gradients of parameters associated with this layer. These two functions are invoked by the <a href="train-one-batch.html">TrainOneBatch</a> function during training. Hence, they should be consistent with the <tt>TrainOneBatch</tt> function. Particularly, for feed-forward and RNN models, they are trained using <a href="train-one-batch.html#back-propagation">BP algorithm</a>, which requires each layer&#x2019;s <tt>ComputeFeature</tt> function to compute <tt>data_</tt> based on source layers, and requires each layer&#x2019;s <tt>ComputeGradient</tt> to compute gradients of parameters and source layers&#x2019; <tt>grad_</tt>. For energy models, e.g., RBM, they are trained by <a href="train-one-batch.html#contrastive-divergence">CD algorithm</a>, which requires each layer&#x2019;s <tt>ComputeFeature</tt> f
 unction to compute the feature vectors for the positive phase or negative phase depending on the <tt>phase</tt> argument, and requires the <tt>ComputeGradient</tt> function to only compute parameter gradients. For some layers, e.g., loss layer or output layer, they can put the loss or prediction result into the <tt>metric</tt> argument, which will be averaged and displayed periodically.</p></div></div>
 <div class="section">
 <h3><a name="Implementing_a_new_Layer_subclass"></a>Implementing a new Layer subclass</h3>
-<p>Users can extend the base layer class to implement their own feature transformation logics as long as the two virtual functions are overridden to be consistent with the <tt>TrainOneBatch</tt> function. The <tt>Setup</tt> function may also be overridden to read specific layer configuration.</p>
+<p>Users can extend the Layer class or other subclasses to implement their own feature transformation logics as long as the two virtual functions are overridden to be consistent with the <tt>TrainOneBatch</tt> function. The <tt>Setup</tt> function may also be overridden to read specific layer configuration.</p>
 <div class="section">
 <h4><a name="Layer_specific_protocol_message"></a>Layer specific protocol message</h4>
 <p>To implement a new layer, the first step is to define the layer specific configuration. Suppose the new layer is <tt>FooLayer</tt>, the layer specific google protocol message <tt>FooLayerProto</tt> should be defined as</p>
@@ -840,7 +854,7 @@ int a = conf.a();
 <div class="source"><pre class="prettyprint">layer {
   name: &quot;foo&quot;
   user_type: &quot;kFooLayer&quot;  # Note user_type of user-defined layers is string
-  [singa.foo_conf] {      # Note there is a pair of [] for extension fields
+  [foo_conf] {      # Note there is a pair of [] for extension fields
     a: 10
   }
 }
@@ -850,7 +864,7 @@ int a = conf.a();
 <p>The new layer subclass can be implemented like the built-in layer subclasses.</p>
 
 <div class="source">
-<div class="source"><pre class="prettyprint">class FooLayer : public Layer {
+<div class="source"><pre class="prettyprint">class FooLayer : public singa::Layer {
  public:
   void Setup(const LayerProto&amp; proto, int npartitions = 1) override;
   void ComputeFeature(Phase phase, Metric* perf) override;
@@ -866,9 +880,9 @@ int a = conf.a();
 <p>The newly defined layer should be registered in <a class="externalLink" href="http://singa.incubator.apache.org/docs/programming-guide">main.cc</a> by adding</p>
 
 <div class="source">
-<div class="source"><pre class="prettyprint">driver.RegisterLayer&lt;FooLayer&gt;(&quot;kFooLayer&quot;); // &quot;kFooLayer&quot; should be matched to layer configurations in job.conf.
+<div class="source"><pre class="prettyprint">driver.RegisterLayer&lt;FooLayer, std::string&gt;(&quot;kFooLayer&quot;); // &quot;kFooLayer&quot; should be matched to layer configurations in job.conf.
 </pre></div></div>
-<p>After that, the <a class="externalLink" href="http://singa.incubator.apache.org/docs/neural-net">NeuralNet</a> can create instances of the new Layer subclass.</p></div></div></div>
+<p>After that, the <a href="neural-net.html">NeuralNet</a> can create instances of the new Layer subclass.</p></div></div></div>
                   </div>
             </div>
           </div>

Modified: websites/staging/singa/trunk/content/docs/lmdb.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/lmdb.html (original)
+++ websites/staging/singa/trunk/content/docs/lmdb.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; </title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           

Modified: websites/staging/singa/trunk/content/docs/mlp.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/mlp.html (original)
+++ websites/staging/singa/trunk/content/docs/mlp.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; </title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           

Modified: websites/staging/singa/trunk/content/docs/model-config.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/model-config.html (original)
+++ websites/staging/singa/trunk/content/docs/model-config.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; Model Configuration</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           

Modified: websites/staging/singa/trunk/content/docs/neural-net.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/neural-net.html (original)
+++ websites/staging/singa/trunk/content/docs/neural-net.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; Neural Net</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           
@@ -483,6 +489,7 @@
         <div id="bodyColumn"  class="span10" >
                                   
             <h1>Neural Net</h1>
+<hr />
 <p><tt>NeuralNet</tt> in SINGA represents an instance of user&#x2019;s neural net model. As the neural net typically consists of a set of layers, <tt>NeuralNet</tt> comprises a set of unidirectionally connected <a class="externalLink" href="http://singa.incubator.apache.org/docs/layer">Layer</a>s. This page describes how to convert an user&#x2019;s neural net into the configuration of <tt>NeuralNet</tt>.</p>
 <p><img src="http://singa.incubator.apache.org/images/model-category.png" align="center" width="200px" alt="" /> <span><b>Figure 1 - Categorization of popular deep learning models.</b></span></p>
 <div class="section">

Modified: websites/staging/singa/trunk/content/docs/neuralnet-partition.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/neuralnet-partition.html (original)
+++ websites/staging/singa/trunk/content/docs/neuralnet-partition.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; Neural Net Partition</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           

Modified: websites/staging/singa/trunk/content/docs/overview.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/overview.html (original)
+++ websites/staging/singa/trunk/content/docs/overview.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; Introduction</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           

Modified: websites/staging/singa/trunk/content/docs/param.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/param.html (original)
+++ websites/staging/singa/trunk/content/docs/param.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; Parameters</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           
@@ -483,6 +489,7 @@
         <div id="bodyColumn"  class="span10" >
                                   
             <h1>Parameters</h1>
+<hr />
 <p>A <tt>Param</tt> object in SINGA represents a set of parameters, e.g., a weight matrix or a bias vector. <i>Basic user guide</i> describes how to configure for a <tt>Param</tt> object, and <i>Advanced user guide</i> provides details on implementing users&#x2019; parameter initialization methods.</p>
 <div class="section">
 <h2><a name="Basic_user_guide"></a>Basic user guide</h2>

Modified: websites/staging/singa/trunk/content/docs/programmer-guide.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/programmer-guide.html (original)
+++ websites/staging/singa/trunk/content/docs/programmer-guide.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; Programmer Guide</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           
@@ -484,79 +490,76 @@
                                   
             <h1>Programmer Guide</h1>
 <hr />
-<p>This guide provides instructions of implementing a new model and submitting the training job. The programming model is made almost transparent to the underlying distributed environment. Hence users do not need to worry much about the communication and synchronization of nodes, which is discussed in <a href="architecture.html">architecture</a> in details.</p>
-<div class="section">
-<div class="section">
-<h3><a name="Deep_learning_training"></a>Deep learning training</h3>
-<p>Deep learning is labeled as a feature learning technique, which usually consists of multiple layers. Each layer is associated with a feature transformation function. After going through all layers, the raw input feature (e.g., pixels of images) would be converted into a high-level feature that is easier for tasks like classification.</p>
-<p>Training a deep learning model is to find the optimal parameters involved in the transformation functions that generates good features for specific tasks. The goodness of a set of parameters is measured by a loss function, e.g., <a class="externalLink" href="https://en.wikipedia.org/wiki/Cross_entropy">Cross-Entropy Loss</a>. Since the loss functions are usually non-linear and non-convex, it is difficult to get a closed form solution. Normally, people uses the SGD algorithm which randomly initializes the parameters and then iteratively update them to reduce the loss.</p></div>
-<div class="section">
-<h3><a name="Steps_to_submit_a_training_job"></a>Steps to submit a training job</h3>
-<p>SINGA uses the stochastic gradient descent (SGD) algorithm to train parameters of deep learning models. For each SGD iteration, there is a <a href="architecture.html">Worker</a> computing gradients of parameters from the NeuralNet and a <a href="">Updater</a> updating parameter values based on gradients. SINGA has implemented three algorithms for gradient calculation, namely Back propagation algorithm for feed-forward models, back-propagation through time for recurrent neural networks and contrastive divergence for energy models like RBM and DBM. Variant SGD updaters are also provided, including <a class="externalLink" href="http://arxiv.org/pdf/1212.5701v1.pdf">AdaDelta</a>, <a class="externalLink" href="http://www.magicbroom.info/Papers/DuchiHaSi10.pdf">AdaGrad</a>, <a class="externalLink" href="http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf">RMSProp</a>, <a class="externalLink" href="http://scholar.google.com/citations?view_op=view_citation&amp;hl=en&a
 mp;user=DJ8Ep8YAAAAJ&amp;citation_for_view=DJ8Ep8YAAAAJ:hkOj_22Ku90C">Nesterov</a>.</p>
-<p>Consequently, what a user needs to do to submit a training job is</p>
+<p>To submit a training job, users must provide the configuration of the four components shown in Figure 1:</p>
 
-<ol style="list-style-type: decimal">
+<ul>
   
-<li>
-<p><a href="data.html">Prepare the data</a> for training, validation and test.</p></li>
+<li>a <a href="neural-net.html">NeuralNet</a> describing the neural net structure with the detailed layer setting and their connections;</li>
   
-<li>
-<p><a href="layer.html">Implement the new Layers</a> to support specific feature transformations  required in the new model.</p></li>
+<li>a <a href="train-one-batch.html">TrainOneBatch</a> algorithm which is tailored for different model categories;</li>
   
-<li>
-<p>Configure the training job including the <a href="architecture.html">cluster setting</a>  and <a href="model-config.html">model configuration</a></p></li>
-</ol></div>
+<li>an <a href="updater.html">Updater</a> defining the protocol for updating parameters at the server side;</li>
+  
+<li>a <a href="distributed-training.html">Cluster Topology</a> specifying the distributed architecture of workers and servers.</li>
+</ul>
+<p>The <i>Basic user guide</i> section describes how to submit a training job using built-in components; while the <i>Advanced user guide</i> section presents details on writing user&#x2019;s own main function to register components implemented by themselves. In addition, the training data must be prepared, which has the same <a href="data.html">process</a> for both advanced users and basic users.</p>
+<p><img src="../images/overview.png" align="center" width="400px" alt="" /> <span><b>Figure 1 - SINGA overview.</b></span></p>
+<div class="section">
+<h2><a name="Basic_user_guide"></a>Basic user guide</h2>
+<p>Users can use the default main function provided by SINGA to submit the training job. For this case, a job configuration file written as a google protocol buffer message for the <a href="../api/classsinga_1_1JobProto.html">JobProto</a> must be provided in the command line,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">./bin/singa-run.sh -conf &lt;path to job conf&gt; [-resume]
+</pre></div></div>
+<p><tt>-resume</tt> is for continuing the training from last <a href="checkpoint.html">checkpoint</a>. The <a href="mlp.html">MLP</a> and <a href="cnn.html">CNN</a> examples use built-in components. Please read the corresponding pages for their job configuration files. The subsequent pages will illustrate the details on each component of the configuration.</p></div>
 <div class="section">
-<h3><a name="Driver_program"></a>Driver program</h3>
-<p>Each training job has a driver program that</p>
+<h2><a name="Advanced_user_guide"></a>Advanced user guide</h2>
+<p>If a user&#x2019;s model contains some user-defined components, e.g., <a href="updater.html">Updater</a>, he has to write a main function to register these components. It is similar to Hadoop&#x2019;s main function. Generally, the main function should</p>
 
 <ul>
   
 <li>
-<p>registers the layers implemented by the user and,</p></li>
+<p>initialize SINGA, e.g., setup logging.</p></li>
+  
+<li>
+<p>register user-defined components.</p></li>
   
 <li>
-<p>submit the job by providing the job configuration.</p></li>
+<p>create and pass the job configuration to SINGA driver</p></li>
 </ul>
-<p>An example driver program is like</p>
+<p>An example main function is like</p>
 
 <div class="source">
-<div class="source"><pre class="prettyprint">#include &quot;singa.h&quot;
-#include &quot;user-layer.h&quot;  // header for user defined layers
-
-DEFINE_int32(job, -1, &quot;Job ID&quot;);  // job ID generated by the SINGA script
-DEFINE_string(workspace, &quot;examples/mnist/&quot;, &quot;workspace of the training job&quot;);
-DEFINE_bool(resume, false, &quot;resume from checkpoint&quot;);
+<div class="source"><pre class="prettyprint">#include &lt;string&gt;
+#include &quot;singa.h&quot;
+#include &quot;user.h&quot;  // header for user code
 
 int main(int argc, char** argv) {
-  google::InitGoogleLogging(argv[0]);
-  gflags::ParseCommandLineFlags(&amp;argc, &amp;argv, true);
-
-  // register all user defined layers in user-layer.h
-  Register(kFooLayer, FooLayer);
+  singa::Driver driver;
+  driver.Init(argc, argv);
+  bool resume;
+  // parse resume option from argv.
+
+  // register user defined layers
+  driver.RegisterLayer&lt;FooLayer, std::string&gt;(&quot;kFooLayer&quot;);
+  // register user defined updater
+  driver.RegisterUpdater&lt;FooUpdater, std::string&gt;(&quot;kFooUpdater&quot;);
   ...
+  auto jobConf = driver.job_conf();
+  //  update jobConf
 
-  JobProto jobConf;
-  // read job configuration from text conf file
-  ReadProtoFromTextFile(&amp;jobConf, FLAGS_workspace + &quot;/job.conf&quot;);
-  singa::SubmitJob(FLAGS_job, FLAGS_resume, jobConf);
+  driver.Submit(resume, jobConf);
+  return 0;
 }
 </pre></div></div>
-<p>Users can also configure the job in the driver program instead of writing the configuration file</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">  JobProto jobConf;
-  jobConf.set_job_name(&quot;my singa job&quot;);
-  ... // configure cluster and model
-  singa::SubmitJob(FLAGS_job, FLAGS_resume, jobConf);
-</pre></div></div>
+<p>The Driver class&#x2019; <tt>Init</tt> method will load a job configuration file provided by users as a command line argument (<tt>-conf &lt;job conf&gt;</tt>). It contains at least the cluster topology and returns the <tt>jobConf</tt> for users to update or fill in configurations of neural net, updater, etc. If users define subclasses of Layer, Updater, Worker and Param, they should register them through the driver. Finally, the job configuration is submitted to the driver which starts the training.</p>
 <p>We will provide helper functions to make the configuration easier in the future, like <a class="externalLink" href="https://github.com/fchollet/keras">keras</a>.</p>
-<p>Compile and link the driver program with singa library to generate an executable file, e.g., with name <tt>mysinga</tt>. To submit the job, just pass the path of the executable and the workspace to the singa job submission script</p>
+<p>Users need to compile and link their code (e.g., layer implementations and the main file) with SINGA library (<i>.libs/libsinga.so</i>) to generate an executable file, e.g., with name <i>mysinga</i>. To launch the program, users just pass the path of the <i>mysinga</i> and base job configuration to <i>./bin/singa-run.sh</i>.</p>
 
 <div class="source">
-<div class="source"><pre class="prettyprint">./bin/singa-run.sh &lt;path to mysinga&gt; -workspace=&lt;my job workspace&gt;
+<div class="source"><pre class="prettyprint">./bin/singa-run.sh -conf &lt;path to job conf&gt; -exec &lt;path to mysinga&gt; [other arguments]
 </pre></div></div>
-<p>The default driver (i.e., main.cc) will be complied and linked to generate <tt>singa</tt>, which is used as the default executable file (no need to put it in the command line)</p></div></div>
+<p>The <a href="rnn.html">RNN application</a> provides a full example of implementing the main function for training a specific RNN model.</p></div>
                   </div>
             </div>
           </div>

Modified: websites/staging/singa/trunk/content/docs/programming-guide.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/programming-guide.html (original)
+++ websites/staging/singa/trunk/content/docs/programming-guide.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; Programming Guide</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           

Modified: websites/staging/singa/trunk/content/docs/quick-start.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/quick-start.html (original)
+++ websites/staging/singa/trunk/content/docs/quick-start.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; Quick Start</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           
@@ -483,9 +489,10 @@
         <div id="bodyColumn"  class="span10" >
                                   
             <h1>Quick Start</h1>
+<hr />
 <div class="section">
 <h2><a name="SINGA_setup"></a>SINGA setup</h2>
-<p>Please refer to the <a class="externalLink" href="http://singa.incubator.apache.org/docs/installation}">installation</a> page for guidance on installing SINGA.</p>
+<p>Please refer to the <a href="installation.html">installation</a> page for guidance on installing SINGA.</p>
 <div class="section">
 <h3><a name="Starting_Zookeeper"></a>Starting Zookeeper</h3>
 <p>SINGA uses <a class="externalLink" href="https://zookeeper.apache.org/">zookeeper</a> to coordinate the training. Please make sure the zookeeper service is started before running SINGA.</p>
@@ -505,10 +512,9 @@ cd  SINGA_ROOT
 <div class="section">
 <h2><a name="Running_in_standalone_mode"></a>Running in standalone mode</h2>
 <p>Running SINGA in standalone mode is on the contrary of running it using cluster managers like <a class="externalLink" href="http://mesos.apache.org/">Mesos</a> or <a class="externalLink" href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html">YARN</a>.</p>
-<p>{% comment %} For standalone mode, users have to manage the resources manually. For instance, they have to prepare a host file containing all running nodes. There is no restriction on CPU and memory resources, hence SINGA consumes as much CPU and memory resources as it needs. {% endcomment %}</p>
 <div class="section">
 <h3><a name="Training_on_a_single_node"></a>Training on a single node</h3>
-<p>For single node training, one process will be launched to run SINGA at local host. We train the <a class="externalLink" href="http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks">CNN model</a> over the <a class="externalLink" href="http://www.cs.toronto.edu/~kriz/cifar.html">CIFAR-10</a> dataset as an example. The hyper-parameters are set following <a class="externalLink" href="https://code.google.com/p/cuda-convnet/">cuda-convnet</a>. More details is available at <a class="externalLink" href="http://singa.incubator.apache.org/docs/cnn">CNN example</a>.</p>
+<p>For single node training, one process will be launched to run SINGA at local host. We train the <a class="externalLink" href="http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks">CNN model</a> over the <a class="externalLink" href="http://www.cs.toronto.edu/~kriz/cifar.html">CIFAR-10</a> dataset as an example. The hyper-parameters are set following <a class="externalLink" href="https://code.google.com/p/cuda-convnet/">cuda-convnet</a>. More details is available at <a href="cnn.html">CNN example</a>.</p>
 <div class="section">
 <h4><a name="Preparing_data_and_job_configuration"></a>Preparing data and job configuration</h4>
 <p>Download the dataset and create the data shards for training and testing.</p>
@@ -519,7 +525,7 @@ make download
 make create
 </pre></div></div>
 <p>A training dataset and a test dataset are created under <i>cifar10-train-shard</i> and <i>cifar10-test-shard</i> folder respectively. An <i>image_mean.bin</i> file is also generated, which contains the feature mean of all images.</p>
-<p>Since all code used for training this CNN model is provided by SINGA as built-in implementation, there is no need to write any code. Instead, users just execute the running script (<i>../../bin/singa-run.sh</i>) by providing the job configuration file (<i>job.conf</i>). To code in SINGA, please refer to the <a class="externalLink" href="http://singa.incubator.apache.org/docs/programming-guide">programming guide</a>.</p></div>
+<p>Since all code used for training this CNN model is provided by SINGA as built-in implementation, there is no need to write any code. Instead, users just execute the running script (<i>../../bin/singa-run.sh</i>) by providing the job configuration file (<i>job.conf</i>). To code in SINGA, please refer to the <a href="programming-guide.html">programming guide</a>.</p></div>
 <div class="section">
 <h4><a name="Training_without_parallelism"></a>Training without parallelism</h4>
 <p>By default, the cluster topology has a single worker and a single server. In other words, neither the training data nor the neural net is partitioned.</p>
@@ -558,7 +564,7 @@ cluster {
   workspace: &quot;examples/cifar10/&quot;
 }
 </pre></div></div>
-<p>In SINGA, <a class="externalLink" href="http://singa.incubator.apache.org/docs/architecture">asynchronous training</a> is enabled by launching multiple worker groups. For example, we can change the original <i>job.conf</i> to have two worker groups as shown above. By default, each worker group has one worker. Since one process is set to contain two workers. The two worker groups will run in the same process. Consequently, they run the in-memory <a class="externalLink" href="http://singa.incubator.apache.org/docs/frameworks">Downpour</a> training framework. Users do not need to split the dataset explicitly for each worker (group); instead, they can assign each worker (group) a random offset to the start of the dataset. The workers would run as on different data partitions.</p>
+<p>In SINGA, <a href="architecture.html">asynchronous training</a> is enabled by launching multiple worker groups. For example, we can change the original <i>job.conf</i> to have two worker groups as shown above. By default, each worker group has one worker. Since one process is set to contain two workers. The two worker groups will run in the same process. Consequently, they run the in-memory <a href="frameworks.html">Downpour</a> training framework. Users do not need to split the dataset explicitly for each worker (group); instead, they can assign each worker (group) a random offset to the start of the dataset. The workers would run as on different data partitions.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint"># job.conf
@@ -590,7 +596,7 @@ cluster {
   workspace: &quot;examples/cifar10/&quot;
 }
 </pre></div></div>
-<p>In SINGA, <a class="externalLink" href="http://singa.incubator.apache.org/docs/architecture">asynchronous training</a> is enabled by launching multiple workers within one worker group. For instance, we can change the original <i>job.conf</i> to have two workers in one worker group as shown above. The workers will run synchronously as they are from the same worker group. This framework is the in-memory <a class="externalLink" href="http://singa.incubator.apache.org/docs/frameworks">sandblaster</a>. The model is partitioned among the two workers. In specific, each layer is sliced over the two workers. The sliced layer is the same as the original layer except that it only has <tt>B/g</tt> feature instances, where <tt>B</tt> is the number of instances in a mini-batch, <tt>g</tt> is the number of workers in a group. It is also possible to partition the layer (or neural net) using <a class="externalLink" href="http://singa.incubator.apache.org/docs/neural-net">other schemes</a>. All ot
 her settings are the same as running without partitioning</p>
+<p>In SINGA, <a href="architecture.html">asynchronous training</a> is enabled by launching multiple workers within one worker group. For instance, we can change the original <i>job.conf</i> to have two workers in one worker group as shown above. The workers will run synchronously as they are from the same worker group. This framework is the in-memory <a href="frameworks.html">sandblaster</a>. The model is partitioned among the two workers. In specific, each layer is sliced over the two workers. The sliced layer is the same as the original layer except that it only has <tt>B/g</tt> feature instances, where <tt>B</tt> is the number of instances in a mini-batch, <tt>g</tt> is the number of workers in a group. It is also possible to partition the layer (or neural net) using <a href="neural-net.html">other schemes</a>. All other settings are the same as running without partitioning</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">./bin/singa-run.sh -conf examples/cifar10/job.conf
@@ -615,10 +621,10 @@ logbase-a02
 </pre></div></div></div></div>
 <div class="section">
 <h2><a name="Running_with_Mesos"></a>Running with Mesos</h2>
-<p><i>in working</i>&#x2026;</p></div>
+<p><i>working</i>&#x2026;</p></div>
 <div class="section">
 <h2><a name="Where_to_go_next"></a>Where to go next</h2>
-<p>The <a class="externalLink" href="http://singa.incubator.apache.org/docs/programming-guide">programming guide</a> pages will describe how to submit a training job in SINGA.</p></div>
+<p>The <a href="programming-guide.html">programming guide</a> pages will describe how to submit a training job in SINGA.</p></div>
                   </div>
             </div>
           </div>

Modified: websites/staging/singa/trunk/content/docs/rbm.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/rbm.html (original)
+++ websites/staging/singa/trunk/content/docs/rbm.html Fri Sep 18 15:11:53 2015
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2015-09-14 
+ | Generated by Apache Maven Doxia at 2015-09-18 
  | Rendered using Apache Maven Fluido Skin 1.4
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20150914" />
+    <meta name="Date-Revision-yyyymmdd" content="20150918" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Apache SINGA &#x2013; RBM Example</title>
     <link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -20,7 +20,13 @@
   
     <script type="text/javascript" src="../js/apache-maven-fluido-1.4.min.js"></script>
 
-    
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+          
                   </head>
         <body class="topBarEnabled">
           
@@ -483,14 +489,15 @@
         <div id="bodyColumn"  class="span10" >
                                   
             <h1>RBM Example</h1>
+<hr />
 <p>This example uses SINGA to train 4 RBM models and one auto-encoder model over the <a class="externalLink" href="http://yann.lecun.com/exdb/mnist/">MNIST dataset</a>. The auto-encoder model is trained to reduce the dimensionality of the MNIST image feature. The RBM models are trained to initialize parameters of the auto-encoder model. This example application is from <a class="externalLink" href="http://www.cs.toronto.edu/~hinton/science.pdf">Hinton&#x2019;s science paper</a>.</p>
 <div class="section">
 <h2><a name="Running_instructions"></a>Running instructions</h2>
 <p>Running scripts are provided in <i>SINGA_ROOT/examples/rbm</i> folder.</p>
-<p>The MNIST dataset has 70,000 handwritten digit images. The <a class="externalLink" href="http://singa.incubator.apache.org/docs/data">data preparation</a> page has details on converting this dataset into SINGA recognizable format (i.e., <a class="externalLink" href="http://singa.incubator.apache.org/api/classsinga_1_1DataShard.html">DataShard</a>). Users can simply run the following commands to download and convert the dataset.</p>
+<p>The MNIST dataset has 70,000 handwritten digit images. The <a href="data.html">data preparation</a> page has details on converting this dataset into SINGA recognizable format (i.e., <a href="../api/classsinga_1_1DataShard.html">DataShard</a>). Users can simply run the following commands to download and convert the dataset.</p>
 
 <div class="source">
-<div class="source"><pre class="prettyprint"># at SINGA_ROOT/examples/rbm/
+<div class="source"><pre class="prettyprint"># at SINGA_ROOT/examples/mnist/
 $ cp Makefile.example Makefile
 $ make download
 $ make create
@@ -513,7 +520,7 @@ $ ./bin/singa-run.sh -conf examples/rbm/
 <h2><a name="Training_details"></a>Training details</h2>
 <div class="section">
 <h3><a name="RBM0"></a>RBM0</h3>
-<p><img src="http://singa.incubator.apache.org/images/RBM0_new.PNG" align="center" width="200px" alt="" /> <span><b>Figure 1 - RBM0.</b></span></p>
+<p><img src="../images/RBM0_new.PNG" align="center" width="200px" alt="" /> <span><b>Figure 1 - RBM0.</b></span></p>
 <p>The neural net structure for training RBM0 is shown in Figure 1. The data layer and parser layer provides features for training RBM0. The visible layer (connected with parser layer) of RBM0 accepts the image feature (784 dimension). The hidden layer is set to have 1000 neurons (units). These two layers are configured as,</p>
 
 <div class="source">
@@ -522,8 +529,8 @@ $ ./bin/singa-run.sh -conf examples/rbm/
   type: kRBMVis
   srclayers:&quot;mnist&quot;
   srclayers:&quot;RBMHid&quot;
-  rbmvis_conf{
-    num_output: 1000
+  rbm_conf{
+    hdim: 1000
   }
   param{
     name: &quot;w0&quot;
@@ -534,7 +541,7 @@ $ ./bin/singa-run.sh -conf examples/rbm/
     }
   }
   param{
-    name: &quot;b0&quot;
+    name: &quot;b01&quot;
     init{
       type: kConstant
       value: 0.0
@@ -546,15 +553,15 @@ layer{
   name: &quot;RBMHid&quot;
   type: kRBMHid
   srclayers:&quot;RBMVis&quot;
-  rbmhid_conf{
-    hid_dim: 1000
+  rbm_conf{
+    hdim: 1000
   }
   param{
     name: &quot;w0_&quot;
     share_from: &quot;w0&quot;
   }
   param{
-    name: &quot;b1&quot;
+    name: &quot;b02&quot;
     init{
       type: kConstant
       value: 0.0
@@ -562,14 +569,14 @@ layer{
   }
 }
 </pre></div></div>
-<p>For RBM, the weight matrix is shared by the visible and hidden layers. For instance, <tt>w0</tt> is shared by <tt>vis</tt> and <tt>hid</tt> layers shown in Figure 1. In SINGA, we can configure the <tt>share_from</tt> field to enable <a class="externalLink" href="http://singa.incubator.apache.org/docs/param">parameter sharing</a> as shown above for the param <tt>w0</tt> and <tt>w0_</tt>.</p>
-<p><a class="externalLink" href="http://singa.incubator.apache.org/docs/train-one-batch/#contrastive-divergence">Contrastive Divergence</a> is configured as the algorithm for <a class="externalLink" href="http://singa.incubator.apache.org/docs/train-one-batch">TrainOneBatch</a>. Following Hinton&#x2019;s paper, we configure the <a class="externalLink" href="http://singa.incubator.apache.org/docs/updater/">updating protocol</a> as follows,</p>
+<p>For RBM, the weight matrix is shared by the visible and hidden layers. For instance, <tt>w0</tt> is shared by <tt>vis</tt> and <tt>hid</tt> layers shown in Figure 1. In SINGA, we can configure the <tt>share_from</tt> field to enable <a href="param.html">parameter sharing</a> as shown above for the param <tt>w0</tt> and <tt>w0_</tt>.</p>
+<p><a href="train-one-batch.html#contrastive-divergence">Contrastive Divergence</a> is configured as the algorithm for <a href="train-one-batch.html">TrainOneBatch</a>. Following Hinton&#x2019;s paper, we configure the <a href="updater.html">updating protocol</a> as follows,</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint"># Updater Configuration
 updater{
   type: kSGD
-  momentum: 0.9
+  momentum: 0.2
   weight_decay: 0.0002
   learning_rate{
     base_lr: 0.1
@@ -581,13 +588,13 @@ updater{
 
 <div class="source">
 <div class="source"><pre class="prettyprint">cluster {
-  workspace: &quot;SINGA_ROOT/rbm0/&quot;
+  workspace: &quot;examples/rbm/rbm0/&quot;
 }
 </pre></div></div>
-<p>Then SINGA will <a class="externalLink" href="http://singa.incubator.apache.org/docs/checkpoint">checkpoint the parameters</a> into <i>SINGA_ROOT/rbm0/</i>.</p></div>
+<p>Then SINGA will <a href="checkpoint.html">checkpoint the parameters</a> into <i>examples/rbm/rbm0/</i>.</p></div>
 <div class="section">
 <h3><a name="RBM1"></a>RBM1</h3>
-<p><img src="http://singa.incubator.apache.org/images/RBM1_new.PNG" align="center" width="200px" alt="" /> <span><b>Figure 2 - RBM1.</b></span></p>
+<p><img src="../images/RBM1_new.PNG" align="center" width="200px" alt="" /> <span><b>Figure 2 - RBM1.</b></span></p>
 <p>Figure 2 shows the net structure of training RBM1. The visible units of RBM1 accept the output from the Sigmoid1 layer. The Inner1 layer is a <tt>InnerProductLayer</tt> whose parameters are set to the <tt>w0</tt> and <tt>b1</tt> learned from RBM0. The neural net configuration is (with layers for data layer and parser layer omitted).</p>
 
 <div class="source">
@@ -598,12 +605,8 @@ updater{
   innerproduct_conf{
     num_output: 1000
   }
-  param{
-    name: &quot;w0&quot;
-  }
-  param{
-    name: &quot;b1&quot;
-  }
+  param{ name: &quot;w0&quot; }
+  param{ name: &quot;b02&quot;}
 }
 
 layer{
@@ -615,25 +618,18 @@ layer{
 layer{
   name: &quot;RBMVis&quot;
   type: kRBMVis
-  srclayers:&quot;sigmoid1&quot;
+  srclayers:&quot;Sigmoid1&quot;
   srclayers:&quot;RBMHid&quot;
-  rbmvis_conf{
-    num_output: 500
+  rbm_conf{
+    hdim: 500
   }
   param{
     name: &quot;w1&quot;
-    init{
-    type: kGaussian
-    mean: 0.0
-    std: 0.1
-    }
+    ...
   }
   param{
-    name: &quot;b2&quot;
-    init{
-    type: kConstant
-    value: 0.0
-    }
+    name: &quot;b11&quot;
+    ...
   }
 }
 
@@ -641,38 +637,35 @@ layer{
   name: &quot;RBMHid&quot;
   type: kRBMHid
   srclayers:&quot;RBMVis&quot;
-  rbmhid_conf{
-    hid_dim: 500
+  rbm_conf{
+    hdim: 500
   }
   param{
     name: &quot;w1_&quot;
     share_from: &quot;w1&quot;
   }
   param{
-    name: &quot;b3&quot;
-    init{
-    type: kConstant
-    value: 0.0
-    }
+    name: &quot;b12&quot;
+    ...
   }
 }
 </pre></div></div>
-<p>To load w0 and b1 from RBM0&#x2019;s checkpoint file, we configure the <tt>checkpoint_path</tt> as,</p>
+<p>To load w0 and b02 from RBM0&#x2019;s checkpoint file, we configure the <tt>checkpoint_path</tt> as,</p>
 
 <div class="source">
-<div class="source"><pre class="prettyprint">checkpoint_path: &quot;SINGA_ROOT/rbm0/checkpoint/step6000-worker0.bin&quot;
+<div class="source"><pre class="prettyprint">checkpoint_path: &quot;examples/rbm/rbm0/checkpoint/step6000-worker0.bin&quot;
 cluster{
-  workspace: &quot;SINGA_ROOT/rbm1&quot;
+  workspace: &quot;examples/rbm/rbm1&quot;
 }
 </pre></div></div>
-<p>The workspace is changed for checkpointing w1, b2 and b3 into <i>SINGA_ROOT/rbm1/</i>.</p></div>
+<p>The workspace is changed for checkpointing <tt>w1</tt>, <tt>b11</tt> and <tt>b12</tt> into <i>examples/rbm/rbm1/</i>.</p></div>
 <div class="section">
 <h3><a name="RBM2"></a>RBM2</h3>
-<p><img src="http://singa.incubator.apache.org/images/RBM2_new.PNG" align="center" width="200px" alt="" /> <span><b>Figure 3 - RBM2.</b></span></p>
-<p>Figure 3 shows the net structure of training RBM2. In this model, a layer with 250 units is added as the hidden layer of RBM2. The visible units of RBM2 accepts output from Sigmoid2 layer. Parameters of Inner1 and inner2 are set to <tt>w0,b1,w1,b2</tt> which can be load from the checkpoint file of RBM1, i.e., &#x201c;SINGA_ROOT/rbm1/&#x201d;.</p></div>
+<p><img src="../images/RBM2_new.PNG" align="center" width="200px" alt="" /> <span><b>Figure 3 - RBM2.</b></span></p>
+<p>Figure 3 shows the net structure of training RBM2. In this model, a layer with 250 units is added as the hidden layer of RBM2. The visible units of RBM2 accepts output from Sigmoid2 layer. Parameters of Inner1 and Innner2 are set to <tt>w0,b02,w1,b12</tt> which can be load from the checkpoint file of RBM1, i.e., &#x201c;examples/rbm/rbm1/&#x201d;.</p></div>
 <div class="section">
 <h3><a name="RBM3"></a>RBM3</h3>
-<p><img src="http://singa.incubator.apache.org/images/RBM3_new.PNG" align="center" width="200px" alt="" /> <span><b>Figure 4 - RBM3.</b></span></p>
+<p><img src="../images/RBM3_new.PNG" align="center" width="200px" alt="" /> <span><b>Figure 4 - RBM3.</b></span></p>
 <p>Figure 4 shows the net structure of training RBM3. It is similar to Figure 3, but according to <a class="externalLink" href="http://www.cs.toronto.edu/~hinton/science.pdf">Hinton&#x2019;s science paper</a>, the hidden units of the top RBM (RBM3) have stochastic real-valued states drawn from a unit variance Gaussian whose mean is determined by the input from the RBM&#x2019;s logistic visible units. So we add a <tt>gaussian</tt> field in the RBMHid layer to control the sampling distribution (Gaussian or Bernoulli). In addition, this RBM has a much smaller learning rate (0.001). The neural net configuration for the RBM3 and the updating protocol is (with layers for data layer and parser layer omitted),</p>
 
 <div class="source">
@@ -690,34 +683,27 @@ updater{
 layer{
   name: &quot;RBMVis&quot;
   type: kRBMVis
-  srclayers:&quot;sigmoid3&quot;
+  srclayers:&quot;Sigmoid3&quot;
   srclayers:&quot;RBMHid&quot;
-  rbmvis_conf{
-    num_output: 30
+  rbm_conf{
+    hdim: 30
   }
   param{
     name: &quot;w3&quot;
-    init{
-    type: kGaussian
-    mean: 0.0
-    std: 0.1
-    }
- }
- param{
-   name: &quot;b6&quot;
-   init{
-   type: kConstant
-   value: 0.0
-   }
- }
+    ...
+  }
+  param{
+    name: &quot;b31&quot;
+    ...
+  }
 }
 
 layer{
   name: &quot;RBMHid&quot;
   type: kRBMHid
   srclayers:&quot;RBMVis&quot;
-  rbmhid_conf{
-    hid_dim: 30
+  rbm_conf{
+    hdim: 30
     gaussian: true
   }
   param{
@@ -725,19 +711,16 @@ layer{
     share_from: &quot;w3&quot;
   }
   param{
-    name: &quot;b7&quot;
-    init{
-      type: kConstant
-      value: 0.0
-    }
+    name: &quot;b32&quot;
+    ...
   }
 }
 </pre></div></div></div>
 <div class="section">
 <h3><a name="Auto-encoder"></a>Auto-encoder</h3>
 <p>In the fine-tuning stage, the 4 RBMs are &#x201c;unfolded&#x201d; to form encoder and decoder networks that are initialized using the parameters from the previous 4 RBMs.</p>
-<p><img src="http://singa.incubator.apache.org/assets/image/autoencoder_new.PNG" align="center" width="500px" alt="" /> <span><b>Figure 5 - Auto-Encoder.</b></span></p>
-<p>Figure 5 shows the neural net structure for training the auto-encoder. <a class="externalLink" href="http://singa.incubator.apache.org/docs/train-one-batch/">Back propagation (kBP)</a> is configured as the algorithm for <tt>TrainOneBatch</tt>. We use the same cluster configuration as RBM models. For updater, we use <a class="externalLink" href="http://singa.incubator.apache.org/docs/updater#adagradupdater">AdaGrad</a> algorithm with fixed learning rate.</p>
+<p><img src="../images/autoencoder_new.PNG" align="center" width="500px" alt="" /> <span><b>Figure 5 - Auto-Encoder.</b></span></p>
+<p>Figure 5 shows the neural net structure for training the auto-encoder. <a href="train-one-batch.html">Back propagation (kBP)</a> is configured as the algorithm for <tt>TrainOneBatch</tt>. We use the same cluster configuration as RBM models. For updater, we use <a href="updater.html#adagradupdater">AdaGrad</a> algorithm with fixed learning rate.</p>
 
 <div class="source">
 <div class="source"><pre class="prettyprint">### Updater Configuration
@@ -757,9 +740,9 @@ layer{ name:&quot;mnist&quot; }
 layer{
   name: &quot;Inner1&quot;
   param{ name: &quot;w0&quot; }
-  param{ name: &quot;b1&quot; }
+  param{ name: &quot;b02&quot; }
 }
-layer{ name: &quot;sigmoid1&quot; }
+layer{ name: &quot;Sigmoid1&quot; }
 ...
 layer{
   name: &quot;Inner8&quot;
@@ -771,14 +754,15 @@ layer{
     name: &quot;w8&quot;
     share_from: &quot;w1&quot;
   }
-  param{ name: &quot;b0&quot; }
+  param{ name: &quot;b01&quot; }
 }
-layer{ name: &quot;sigmoid8&quot; }
-### Euclidean Loss Layer Configuration
+layer{ name: &quot;Sigmoid8&quot; }
+
+# Euclidean Loss Layer Configuration
 layer{
   name: &quot;loss&quot;
   type:kEuclideanLoss
-  srclayers:&quot;sigmoid8&quot;
+  srclayers:&quot;Sigmoid8&quot;
   srclayers:&quot;mnist&quot;
 }
 </pre></div></div>
@@ -795,9 +779,9 @@ checkpoint_path: &quot;examples/rbm/chec
 <h2><a name="Visualization_Results"></a>Visualization Results</h2>
 
 <div>
-<img src="http://singa.incubator.apache.org/assets/image/rbm-weight.PNG" align="center" width="300px" alt="" />
+<img src="../image/rbm-weight.PNG" align="center" width="300px" alt="" />
 
-<img src="http://singa.incubator.apache.org/assets/image/rbm-feature.PNG" align="center" width="300px" alt="" />
+<img src="../image/rbm-feature.PNG" align="center" width="300px" alt="" />
 <br />
 <span><b>Figure 6 - Bottom RBM weight matrix.</b></span>
 &#160;



Mime
View raw message