singa-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From build...@apache.org
Subject svn commit: r977575 [11/19] - in /websites/staging/singa/trunk/content: ./ community/ develop/ docs/ docs/jp/ docs/kr/ docs/zh/ releases/ v0.1.0/
Date Wed, 13 Jan 2016 03:47:58 GMT
Added: websites/staging/singa/trunk/content/docs/kr/checkpoint.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/kr/checkpoint.html (added)
+++ websites/staging/singa/trunk/content/docs/kr/checkpoint.html Wed Jan 13 03:47:56 2016
@@ -0,0 +1,410 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-01-13 
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20160113" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Apache SINGA &#x2013; CheckPoint</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+      
+    
+    
+  
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+                      
+        
+<meta content="Apache SINGA" name="author"/>
+                      
+        
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+          
+                  </head>
+        <body class="topBarEnabled">
+          
+    
+    
+            
+    
+        
+    <a href="https://github.com/apache/incubator-singa">
+      <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+        src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+        alt="Fork me on GitHub">
+    </a>
+  
+                
+                    
+                
+
+    <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+      <div class="navbar-inner">
+                <div class="container-fluid">
+        <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+        </a>
+                
+                                <ul class="nav">
+                          <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../docs/overview.html"  title="Introduction">Introduction</a>
+</li>
+                  
+                      <li>      <a href="../../docs/quick-start.html"  title="Quick Start">Quick Start</a>
+</li>
+                  
+                      <li>      <a href="../../downloads.html"  title="Downloads">Downloads</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li class="dropdown-submenu">
+                                      <a href="../../docs/index.html"  title="Latest">Latest</a>
+              <ul class="dropdown-menu">
+                                  <li>      <a href="../../docs/installation.html"  title="Installation">Installation</a>
+</li>
+                                  <li>      <a href="../../docs/programmer-guide.html"  title="Programmer Guide">Programmer Guide</a>
+</li>
+                                  <li>      <a href="../../docs/distributed-training.html"  title="Distributed Training">Distributed Training</a>
+</li>
+                                  <li>      <a href="../../docs/data.html"  title="Data Preparation">Data Preparation</a>
+</li>
+                                  <li>      <a href="../../docs/checkpoint.html"  title="Checkpoint">Checkpoint</a>
+</li>
+                                  <li>      <a href="../../docs/test.html"  title="Test">Test</a>
+</li>
+                                  <li>      <a href="../../docs/examples.html"  title="Examples">Examples</a>
+</li>
+                                  <li>      <a href="../../docs/debug.html"  title="Debug">Debug</a>
+</li>
+                              </ul>
+            </li>
+                  
+                      <li>      <a href="../../v0.1.0/index.html"  title="v0.1.0">v0.1.0</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../develop/schedule.html"  title="Schedule">Schedule</a>
+</li>
+                  
+                      <li class="dropdown-submenu">
+                                      <a href="../../develop/how-contribute.html"  title="How to Contribute">How to Contribute</a>
+              <ul class="dropdown-menu">
+                                  <li>      <a href="../../develop/contribute-code.html"  title="Code">Code</a>
+</li>
+                                  <li>      <a href="../../develop/contribute-docs.html"  title="Documentation">Documentation</a>
+</li>
+                              </ul>
+            </li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../community/source-repository.html"  title="Source Repository">Source Repository</a>
+</li>
+                  
+                      <li>      <a href="../../community/mail-lists.html"  title="Mailing Lists">Mailing Lists</a>
+</li>
+                  
+                      <li>      <a href="../../community/issue-tracking.html"  title="Issue Tracking">Issue Tracking</a>
+</li>
+                  
+                      <li>      <a href="../../community/team-list.html"  title="SINGA Team">SINGA Team</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="http://www.apache.org/"  title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+                  
+                      <li>      <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/"  title="NUS Site">NUS Site</a>
+</li>
+                          </ul>
+      </li>
+                  </ul>
+          
+          
+          
+                   
+                      </div>
+          
+        </div>
+      </div>
+    </div>
+    
+        <div class="container-fluid">
+          <div id="banner">
+        <div class="pull-left">
+                                                  <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+                                                                                                <img src="../../images/singa-logo.png"  alt="Apache SINGA"/>
+                </a>
+                      </div>
+        <div class="pull-right">              <div id="bannerRight">
+                                                                                                <img src="../../images/singa-title.png"  alt="Apache SINGA"/>
+                </div>
+      </div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+                
+                    
+                              <li class="">
+                    <a href="../../index.html" title="Apache SINGA">
+        Apache SINGA</a>
+                    <span class="divider">/</span>
+      </li>
+        <li class="active ">CheckPoint</li>
+        
+                
+                    
+      
+                            </ul>
+      </div>
+
+                  
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+                
+                    
+                <ul class="nav nav-list">
+                    <li class="nav-header">Apache SINGA</li>
+                              
+      <li>
+  
+                          <a href="../../docs/overview.html" title="Introduction">
+          <span class="none"></span>
+        Introduction</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../docs/quick-start.html" title="Quick Start">
+          <span class="none"></span>
+        Quick Start</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../downloads.html" title="Downloads">
+          <span class="none"></span>
+        Downloads</a>
+            </li>
+                              <li class="nav-header">Documentaion</li>
+                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
+      <li>
+  
+                          <a href="../../docs/index.html" title="Latest">
+          <span class="icon-chevron-right"></span>
+        Latest</a>
+                  </li>
+                
+      <li>
+  
+                          <a href="../../v0.1.0/index.html" title="v0.1.0">
+          <span class="none"></span>
+        v0.1.0</a>
+            </li>
+                              <li class="nav-header">Development</li>
+                              
+      <li>
+  
+                          <a href="../../develop/schedule.html" title="Schedule">
+          <span class="none"></span>
+        Schedule</a>
+            </li>
+                                                                                    
+      <li>
+  
+                          <a href="../../develop/how-contribute.html" title="How to Contribute">
+          <span class="icon-chevron-right"></span>
+        How to Contribute</a>
+                  </li>
+                              <li class="nav-header">Community</li>
+                              
+      <li>
+  
+                          <a href="../../community/source-repository.html" title="Source Repository">
+          <span class="none"></span>
+        Source Repository</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/mail-lists.html" title="Mailing Lists">
+          <span class="none"></span>
+        Mailing Lists</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/issue-tracking.html" title="Issue Tracking">
+          <span class="none"></span>
+        Issue Tracking</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/team-list.html" title="SINGA Team">
+          <span class="none"></span>
+        SINGA Team</a>
+            </li>
+                              <li class="nav-header">External Links</li>
+                              
+      <li>
+  
+                          <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+          <span class="none"></span>
+        Apache Software Foundation</a>
+            </li>
+                
+      <li>
+  
+                          <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+          <span class="none"></span>
+        NUS Site</a>
+            </li>
+            </ul>
+                
+                    
+                
+          <hr />
+
+           <div id="poweredBy">
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                                                                                                                   <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+        <img class="builtBy"  alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png"    />
+      </a>
+                      </div>
+          </div>
+        </div>
+        
+                        
+        <div id="bodyColumn"  class="span10" >
+                                  
+            <h1>CheckPoint</h1>
+<hr />
+<p>SINGA checkpoints model parameters onto disk periodically according to user configured frequency. By checkpointing model parameters, we can</p>
+
+<ol style="list-style-type: decimal">
+  
+<li>
+<p>resume the training from the last checkpointing. For example, if the program crashes before finishing all training steps, we can continue the training using checkpoint files.</p></li>
+  
+<li>
+<p>use them to initialize a similar model. For example, the parameters from training a RBM model can be used to initialize a <a href="rbm.html">deep auto-encoder</a> model.</p></li>
+</ol>
+<div class="section">
+<h2><a name="Configuration"></a>Configuration</h2>
+<p>Checkpointing is controlled by two configuration fields:</p>
+
+<ul>
+  
+<li><tt>checkpoint_after</tt>, start checkpointing after this number of training steps,</li>
+  
+<li><tt>checkpoint_freq</tt>, frequency of doing checkpointing.</li>
+</ul>
+<p>For example,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"># job.conf
+checkpoint_after: 100
+checkpoint_frequency: 300
+...
+</pre></div></div>
+<p>Checkpointing files are located at <i>WORKSPACE/checkpoint/stepSTEP-workerWORKERID</i>. <i>WORKSPACE</i> is configured in</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">cluster {
+  workspace:
+}
+</pre></div></div>
+<p>For the above configuration, after training for 700 steps, there would be two checkpointing files,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">step400-worker0
+step700-worker0
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Application_-_resuming_training"></a>Application - resuming training</h2>
+<p>We can resume the training from the last checkpoint (i.e., step 700) by,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">./bin/singa-run.sh -conf JOB_CONF -resume
+</pre></div></div>
+<p>There is no change to the job configuration.</p></div>
+<div class="section">
+<h2><a name="Application_-_model_initialization"></a>Application - model initialization</h2>
+<p>We can also use the checkpointing file from step 400 to initialize a new model by configuring the new job as,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"># job.conf
+checkpoint : &quot;WORKSPACE/checkpoint/step400-worker0&quot;
+...
+</pre></div></div>
+<p>If there are multiple checkpointing files for the same snapshot due to model partitioning, all the checkpointing files should be added,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"># job.conf
+checkpoint : &quot;WORKSPACE/checkpoint/step400-worker0&quot;
+checkpoint : &quot;WORKSPACE/checkpoint/step400-worker1&quot;
+...
+</pre></div></div>
+<p>The training command is the same as starting a new job,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">./bin/singa-run.sh -conf JOB_CONF
+</pre></div></div></div>
+                  </div>
+            </div>
+          </div>
+
+    <hr/>
+
+    <footer>
+            <div class="container-fluid">
+                      <div class="row-fluid">
+                                                                          
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+                          </div>
+
+        
+                </div>
+    </footer>
+        </body>
+</html>

Added: websites/staging/singa/trunk/content/docs/kr/cnn.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/kr/cnn.html (added)
+++ websites/staging/singa/trunk/content/docs/kr/cnn.html Wed Jan 13 03:47:56 2016
@@ -0,0 +1,542 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-01-13 
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20160113" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Apache SINGA &#x2013; CNN Example</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+      
+    
+    
+  
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+                      
+        
+<meta content="Apache SINGA" name="author"/>
+                      
+        
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+          
+                  </head>
+        <body class="topBarEnabled">
+          
+    
+    
+            
+    
+        
+    <a href="https://github.com/apache/incubator-singa">
+      <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+        src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+        alt="Fork me on GitHub">
+    </a>
+  
+                
+                    
+                
+
+    <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+      <div class="navbar-inner">
+                <div class="container-fluid">
+        <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+        </a>
+                
+                                <ul class="nav">
+                          <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../docs/overview.html"  title="Introduction">Introduction</a>
+</li>
+                  
+                      <li>      <a href="../../docs/quick-start.html"  title="Quick Start">Quick Start</a>
+</li>
+                  
+                      <li>      <a href="../../downloads.html"  title="Downloads">Downloads</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li class="dropdown-submenu">
+                                      <a href="../../docs/index.html"  title="Latest">Latest</a>
+              <ul class="dropdown-menu">
+                                  <li>      <a href="../../docs/installation.html"  title="Installation">Installation</a>
+</li>
+                                  <li>      <a href="../../docs/programmer-guide.html"  title="Programmer Guide">Programmer Guide</a>
+</li>
+                                  <li>      <a href="../../docs/distributed-training.html"  title="Distributed Training">Distributed Training</a>
+</li>
+                                  <li>      <a href="../../docs/data.html"  title="Data Preparation">Data Preparation</a>
+</li>
+                                  <li>      <a href="../../docs/checkpoint.html"  title="Checkpoint">Checkpoint</a>
+</li>
+                                  <li>      <a href="../../docs/test.html"  title="Test">Test</a>
+</li>
+                                  <li>      <a href="../../docs/examples.html"  title="Examples">Examples</a>
+</li>
+                                  <li>      <a href="../../docs/debug.html"  title="Debug">Debug</a>
+</li>
+                              </ul>
+            </li>
+                  
+                      <li>      <a href="../../v0.1.0/index.html"  title="v0.1.0">v0.1.0</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../develop/schedule.html"  title="Schedule">Schedule</a>
+</li>
+                  
+                      <li class="dropdown-submenu">
+                                      <a href="../../develop/how-contribute.html"  title="How to Contribute">How to Contribute</a>
+              <ul class="dropdown-menu">
+                                  <li>      <a href="../../develop/contribute-code.html"  title="Code">Code</a>
+</li>
+                                  <li>      <a href="../../develop/contribute-docs.html"  title="Documentation">Documentation</a>
+</li>
+                              </ul>
+            </li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../community/source-repository.html"  title="Source Repository">Source Repository</a>
+</li>
+                  
+                      <li>      <a href="../../community/mail-lists.html"  title="Mailing Lists">Mailing Lists</a>
+</li>
+                  
+                      <li>      <a href="../../community/issue-tracking.html"  title="Issue Tracking">Issue Tracking</a>
+</li>
+                  
+                      <li>      <a href="../../community/team-list.html"  title="SINGA Team">SINGA Team</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="http://www.apache.org/"  title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+                  
+                      <li>      <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/"  title="NUS Site">NUS Site</a>
+</li>
+                          </ul>
+      </li>
+                  </ul>
+          
+          
+          
+                   
+                      </div>
+          
+        </div>
+      </div>
+    </div>
+    
+        <div class="container-fluid">
+          <div id="banner">
+        <div class="pull-left">
+                                                  <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+                                                                                                <img src="../../images/singa-logo.png"  alt="Apache SINGA"/>
+                </a>
+                      </div>
+        <div class="pull-right">              <div id="bannerRight">
+                                                                                                <img src="../../images/singa-title.png"  alt="Apache SINGA"/>
+                </div>
+      </div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+                
+                    
+                              <li class="">
+                    <a href="../../index.html" title="Apache SINGA">
+        Apache SINGA</a>
+                    <span class="divider">/</span>
+      </li>
+        <li class="active ">CNN Example</li>
+        
+                
+                    
+      
+                            </ul>
+      </div>
+
+                  
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+                
+                    
+                <ul class="nav nav-list">
+                    <li class="nav-header">Apache SINGA</li>
+                              
+      <li>
+  
+                          <a href="../../docs/overview.html" title="Introduction">
+          <span class="none"></span>
+        Introduction</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../docs/quick-start.html" title="Quick Start">
+          <span class="none"></span>
+        Quick Start</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../downloads.html" title="Downloads">
+          <span class="none"></span>
+        Downloads</a>
+            </li>
+                              <li class="nav-header">Documentaion</li>
+                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
+      <li>
+  
+                          <a href="../../docs/index.html" title="Latest">
+          <span class="icon-chevron-right"></span>
+        Latest</a>
+                  </li>
+                
+      <li>
+  
+                          <a href="../../v0.1.0/index.html" title="v0.1.0">
+          <span class="none"></span>
+        v0.1.0</a>
+            </li>
+                              <li class="nav-header">Development</li>
+                              
+      <li>
+  
+                          <a href="../../develop/schedule.html" title="Schedule">
+          <span class="none"></span>
+        Schedule</a>
+            </li>
+                                                                                    
+      <li>
+  
+                          <a href="../../develop/how-contribute.html" title="How to Contribute">
+          <span class="icon-chevron-right"></span>
+        How to Contribute</a>
+                  </li>
+                              <li class="nav-header">Community</li>
+                              
+      <li>
+  
+                          <a href="../../community/source-repository.html" title="Source Repository">
+          <span class="none"></span>
+        Source Repository</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/mail-lists.html" title="Mailing Lists">
+          <span class="none"></span>
+        Mailing Lists</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/issue-tracking.html" title="Issue Tracking">
+          <span class="none"></span>
+        Issue Tracking</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/team-list.html" title="SINGA Team">
+          <span class="none"></span>
+        SINGA Team</a>
+            </li>
+                              <li class="nav-header">External Links</li>
+                              
+      <li>
+  
+                          <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+          <span class="none"></span>
+        Apache Software Foundation</a>
+            </li>
+                
+      <li>
+  
+                          <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+          <span class="none"></span>
+        NUS Site</a>
+            </li>
+            </ul>
+                
+                    
+                
+          <hr />
+
+           <div id="poweredBy">
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                                                                                                                   <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+        <img class="builtBy"  alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png"    />
+      </a>
+                      </div>
+          </div>
+        </div>
+        
+                        
+        <div id="bodyColumn"  class="span10" >
+                                  
+            <h1>CNN Example</h1>
+<hr />
+<p>Convolutional neural network (CNN) is a type of feed-forward artificial neural network widely used for image and video classification. In this example, we will use a deep CNN model to do image classification for the <a class="externalLink" href="http://www.cs.toronto.edu/~kriz/cifar.html">CIFAR10 dataset</a>.</p>
+<div class="section">
+<h2><a name="Running_instructions"></a>Running instructions</h2>
+<p>Please refer to the <a href="installation.html">installation</a> page for instructions on building SINGA, and the <a href="quick-start.html">quick start</a> for instructions on starting zookeeper.</p>
+<p>We have provided scripts for preparing the training and test dataset in <i>examples/cifar10/</i>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"># in examples/cifar10
+$ cp Makefile.example Makefile
+$ make download
+$ make create
+</pre></div></div>
+<p>We can start the training by</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">./bin/singa-run.sh -conf examples/cifar10/job.conf
+</pre></div></div>
+<p>You should see output like</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">Record job information to /tmp/singa-log/job-info/job-2-20150817-055601
+Executing : ./singa -conf /xxx/incubator-singa/examples/cifar10/job.conf -singa_conf /xxx/incubator-singa/conf/singa.conf -singa_job 2
+E0817 06:56:18.868259 33849 cluster.cc:51] proc #0 -&gt; 192.168.5.128:49152 (pid = 33849)
+E0817 06:56:18.928452 33871 server.cc:36] Server (group = 0, id = 0) start
+E0817 06:56:18.928469 33872 worker.cc:134] Worker (group = 0, id = 0) start
+E0817 06:57:13.657302 33849 trainer.cc:373] Test step-0, loss : 2.302588, accuracy : 0.077900
+E0817 06:57:17.626708 33849 trainer.cc:373] Train step-0, loss : 2.302578, accuracy : 0.062500
+E0817 06:57:24.142645 33849 trainer.cc:373] Train step-30, loss : 2.302404, accuracy : 0.131250
+E0817 06:57:30.813354 33849 trainer.cc:373] Train step-60, loss : 2.302248, accuracy : 0.156250
+E0817 06:57:37.556655 33849 trainer.cc:373] Train step-90, loss : 2.301849, accuracy : 0.175000
+E0817 06:57:44.971276 33849 trainer.cc:373] Train step-120, loss : 2.301077, accuracy : 0.137500
+E0817 06:57:51.801949 33849 trainer.cc:373] Train step-150, loss : 2.300410, accuracy : 0.135417
+E0817 06:57:58.682281 33849 trainer.cc:373] Train step-180, loss : 2.300067, accuracy : 0.127083
+E0817 06:58:05.578366 33849 trainer.cc:373] Train step-210, loss : 2.300143, accuracy : 0.154167
+E0817 06:58:12.518497 33849 trainer.cc:373] Train step-240, loss : 2.295912, accuracy : 0.185417
+</pre></div></div>
+<p>After training some steps (depends on the setting) or the job is finished, SINGA will <a href="checkpoint.html">checkpoint</a> the model parameters.</p></div>
+<div class="section">
+<h2><a name="Details"></a>Details</h2>
+<p>To train a model in SINGA, you need to prepare the datasets, and a job configuration which specifies the neural net structure, training algorithm (BP or CD), SGD update algorithm (e.g. Adagrad), number of training/test steps, etc.</p>
+<div class="section">
+<h3><a name="Data_preparation"></a>Data preparation</h3>
+<p>Before using SINGA, you need to write a program to convert the dataset into a format that SINGA can read. Please refer to the <a href="data.html#example---cifar-dataset">Data Preparation</a> to get details about preparing this CIFAR10 dataset.</p></div>
+<div class="section">
+<h3><a name="Neural_net"></a>Neural net</h3>
+<p>Figure 1 shows the net structure of the CNN model we used in this example, which is set following <a class="externalLink" href="https://code.google.com/p/cuda-convnet/source/browse/trunk/example-layers/layers-18pct.cfg.">Alex</a> The dashed circle represents one feature transformation stage, which generally has four layers as shown in the figure. Sometimes the rectifier layer and normalization layer are omitted or swapped in one stage. For this example, there are 3 such stages.</p>
+<p>Next we follow the guide in <a href="neural-net.html">neural net page</a> and <a href="layer.html">layer page</a> to write the neural net configuration.</p>
+
+<div style="text-align: center">
+<img src="../images/example-cnn.png" style="width: 200px" alt="" /> <br />
+<b>Figure 1 - Net structure of the CNN example.</b></img>
+</div>
+
+<ul>
+  
+<li>
+<p>We configure an input layer to read the training/testing records from a disk file.</p>
+  
+<div class="source">
+<div class="source"><pre class="prettyprint">layer{
+  name: &quot;data&quot;
+  type: kRecordInput
+  store_conf {
+    backend: &quot;kvfile&quot;
+    path: &quot;examples/cifar10/train_data.bin&quot;
+    mean_file: &quot;examples/cifar10/image_mean.bin&quot;
+    batchsize: 64
+    random_skip: 5000
+    shape: 3
+    shape: 32
+    shape: 32
+   }
+   exclude: kTest  # exclude this layer for the testing net
+}
+layer{
+  name: &quot;data&quot;
+  type: kRecordInput
+  store_conf {
+    backend: &quot;kvfile&quot;
+    path: &quot;examples/cifar10/test_data.bin&quot;
+    mean_file: &quot;examples/cifar10/image_mean.bin&quot;
+    batchsize: 100
+    shape: 3
+    shape: 32
+    shape: 32
+   }
+ exclude: kTrain # exclude this layer for the training net
+}
+</pre></div></div></li>
+</ul>
+
+<ul>
+  
+<li>
+<p>We configure layers for the feature transformation as follows (all layers are built-in layers in SINGA; hyper-parameters of these layers are set according to <a class="externalLink" href="https://code.google.com/p/cuda-convnet/source/browse/trunk/example-layers/layers-18pct.cfg">Alex&#x2019;s setting</a>).</p>
+  
+<div class="source">
+<div class="source"><pre class="prettyprint">layer {
+  name: &quot;conv1&quot;
+  type: kConvolution
+  srclayers: &quot;data&quot;
+  convolution_conf {... }
+  ...
+}
+layer {
+  name: &quot;pool1&quot;
+  type: kPooling
+  srclayers: &quot;conv1&quot;
+  pooling_conf {... }
+}
+layer {
+  name: &quot;relu1&quot;
+  type: kReLU
+  srclayers:&quot;pool1&quot;
+}
+layer {
+  name: &quot;norm1&quot;
+  type: kLRN
+  lrn_conf {... }
+  srclayers:&quot;relu1&quot;
+}
+</pre></div></div></li>
+</ul>
+<p>The configurations for another 2 stages are omitted here.</p>
+
+<ul>
+  
+<li>
+<p>There is an <a href="layer.html#innerproductlayer">inner product layer</a> after the 3 transformation stages, which is configured with 10 output units, i.e., the number of total labels. The weight matrix Param is configured with a large weight decay scale to reduce the over-fitting.</p>
+  
+<div class="source">
+<div class="source"><pre class="prettyprint">layer {
+  name: &quot;ip1&quot;
+  type: kInnerProduct
+  srclayers:&quot;pool3&quot;
+  innerproduct_conf {
+    num_output: 10
+  }
+  param {
+    name: &quot;w4&quot;
+    wd_scale:250
+    ...
+  }
+  param {
+    name: &quot;b4&quot;
+    ...
+  }
+}
+</pre></div></div></li>
+  
+<li>
+<p>The last layer is a <a href="layer.html#softmaxloss">Softmax loss layer</a></p>
+  
+<div class="source">
+<div class="source"><pre class="prettyprint">layer{
+  name: &quot;loss&quot;
+  type: kSoftmaxLoss
+  softmaxloss_conf{ topk:1 }
+  srclayers:&quot;ip1&quot;
+  srclayers: &quot;data&quot;
+}
+</pre></div></div></li>
+</ul></div>
+<div class="section">
+<h3><a name="Updater"></a>Updater</h3>
+<p>The <a href="updater.html#updater">normal SGD updater</a> is selected. The learning rate is changed like going down stairs, and is configured using the <a href="updater.html#kfixedstep">kFixedStep</a> type.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">    updater{
+      type: kSGD
+      weight_decay:0.004
+      learning_rate {
+        type: kFixedStep
+        fixedstep_conf:{
+          step:0             # lr for step 0-60000 is 0.001
+          step:60000         # lr for step 60000-65000 is 0.0001
+          step:65000         # lr for step 650000- is 0.00001
+          step_lr:0.001
+          step_lr:0.0001
+          step_lr:0.00001
+        }
+      }
+    }
+</pre></div></div></div>
+<div class="section">
+<h3><a name="TrainOneBatch_algorithm"></a>TrainOneBatch algorithm</h3>
+<p>The CNN model is a feed forward model, thus should be configured to use the <a href="train-one-batch.html#back-propagation">Back-propagation algorithm</a>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">train_one_batch {
+  alg: kBP
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Cluster_setting"></a>Cluster setting</h3>
+<p>The following configuration set a single worker and server for training. <a href="frameworks.html">Training frameworks</a> page introduces configurations of a couple of distributed training frameworks.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">cluster {
+  nworker_groups: 1
+  nserver_groups: 1
+}
+</pre></div></div></div></div>
+                  </div>
+            </div>
+          </div>
+
+    <hr/>
+
+    <footer>
+            <div class="container-fluid">
+                      <div class="row-fluid">
+                                                                          
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+                          </div>
+
+        
+                </div>
+    </footer>
+        </body>
+</html>

Added: websites/staging/singa/trunk/content/docs/kr/code-structure.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/kr/code-structure.html (added)
+++ websites/staging/singa/trunk/content/docs/kr/code-structure.html Wed Jan 13 03:47:56 2016
@@ -0,0 +1,407 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-01-13 
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20160113" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Apache SINGA &#x2013; Code Structure</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+      
+    
+    
+  
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+                      
+        
+<meta content="Apache SINGA" name="author"/>
+                      
+        
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+          
+                  </head>
+        <body class="topBarEnabled">
+          
+    
+    
+            
+    
+        
+    <a href="https://github.com/apache/incubator-singa">
+      <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+        src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+        alt="Fork me on GitHub">
+    </a>
+  
+                
+                    
+                
+
+    <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+      <div class="navbar-inner">
+                <div class="container-fluid">
+        <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+        </a>
+                
+                                <ul class="nav">
+                          <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../docs/overview.html"  title="Introduction">Introduction</a>
+</li>
+                  
+                      <li>      <a href="../../docs/quick-start.html"  title="Quick Start">Quick Start</a>
+</li>
+                  
+                      <li>      <a href="../../downloads.html"  title="Downloads">Downloads</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li class="dropdown-submenu">
+                                      <a href="../../docs/index.html"  title="Latest">Latest</a>
+              <ul class="dropdown-menu">
+                                  <li>      <a href="../../docs/installation.html"  title="Installation">Installation</a>
+</li>
+                                  <li>      <a href="../../docs/programmer-guide.html"  title="Programmer Guide">Programmer Guide</a>
+</li>
+                                  <li>      <a href="../../docs/distributed-training.html"  title="Distributed Training">Distributed Training</a>
+</li>
+                                  <li>      <a href="../../docs/data.html"  title="Data Preparation">Data Preparation</a>
+</li>
+                                  <li>      <a href="../../docs/checkpoint.html"  title="Checkpoint">Checkpoint</a>
+</li>
+                                  <li>      <a href="../../docs/test.html"  title="Test">Test</a>
+</li>
+                                  <li>      <a href="../../docs/examples.html"  title="Examples">Examples</a>
+</li>
+                                  <li>      <a href="../../docs/debug.html"  title="Debug">Debug</a>
+</li>
+                              </ul>
+            </li>
+                  
+                      <li>      <a href="../../v0.1.0/index.html"  title="v0.1.0">v0.1.0</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../develop/schedule.html"  title="Schedule">Schedule</a>
+</li>
+                  
+                      <li class="dropdown-submenu">
+                                      <a href="../../develop/how-contribute.html"  title="How to Contribute">How to Contribute</a>
+              <ul class="dropdown-menu">
+                                  <li>      <a href="../../develop/contribute-code.html"  title="Code">Code</a>
+</li>
+                                  <li>      <a href="../../develop/contribute-docs.html"  title="Documentation">Documentation</a>
+</li>
+                              </ul>
+            </li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../community/source-repository.html"  title="Source Repository">Source Repository</a>
+</li>
+                  
+                      <li>      <a href="../../community/mail-lists.html"  title="Mailing Lists">Mailing Lists</a>
+</li>
+                  
+                      <li>      <a href="../../community/issue-tracking.html"  title="Issue Tracking">Issue Tracking</a>
+</li>
+                  
+                      <li>      <a href="../../community/team-list.html"  title="SINGA Team">SINGA Team</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="http://www.apache.org/"  title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+                  
+                      <li>      <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/"  title="NUS Site">NUS Site</a>
+</li>
+                          </ul>
+      </li>
+                  </ul>
+          
+          
+          
+                   
+                      </div>
+          
+        </div>
+      </div>
+    </div>
+    
+        <div class="container-fluid">
+          <div id="banner">
+        <div class="pull-left">
+                                                  <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+                                                                                                <img src="../../images/singa-logo.png"  alt="Apache SINGA"/>
+                </a>
+                      </div>
+        <div class="pull-right">              <div id="bannerRight">
+                                                                                                <img src="../../images/singa-title.png"  alt="Apache SINGA"/>
+                </div>
+      </div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+                
+                    
+                              <li class="">
+                    <a href="../../index.html" title="Apache SINGA">
+        Apache SINGA</a>
+                    <span class="divider">/</span>
+      </li>
+        <li class="active ">Code Structure</li>
+        
+                
+                    
+      
+                            </ul>
+      </div>
+
+                  
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+                
+                    
+                <ul class="nav nav-list">
+                    <li class="nav-header">Apache SINGA</li>
+                              
+      <li>
+  
+                          <a href="../../docs/overview.html" title="Introduction">
+          <span class="none"></span>
+        Introduction</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../docs/quick-start.html" title="Quick Start">
+          <span class="none"></span>
+        Quick Start</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../downloads.html" title="Downloads">
+          <span class="none"></span>
+        Downloads</a>
+            </li>
+                              <li class="nav-header">Documentaion</li>
+                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
+      <li>
+  
+                          <a href="../../docs/index.html" title="Latest">
+          <span class="icon-chevron-right"></span>
+        Latest</a>
+                  </li>
+                
+      <li>
+  
+                          <a href="../../v0.1.0/index.html" title="v0.1.0">
+          <span class="none"></span>
+        v0.1.0</a>
+            </li>
+                              <li class="nav-header">Development</li>
+                              
+      <li>
+  
+                          <a href="../../develop/schedule.html" title="Schedule">
+          <span class="none"></span>
+        Schedule</a>
+            </li>
+                                                                                    
+      <li>
+  
+                          <a href="../../develop/how-contribute.html" title="How to Contribute">
+          <span class="icon-chevron-right"></span>
+        How to Contribute</a>
+                  </li>
+                              <li class="nav-header">Community</li>
+                              
+      <li>
+  
+                          <a href="../../community/source-repository.html" title="Source Repository">
+          <span class="none"></span>
+        Source Repository</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/mail-lists.html" title="Mailing Lists">
+          <span class="none"></span>
+        Mailing Lists</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/issue-tracking.html" title="Issue Tracking">
+          <span class="none"></span>
+        Issue Tracking</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/team-list.html" title="SINGA Team">
+          <span class="none"></span>
+        SINGA Team</a>
+            </li>
+                              <li class="nav-header">External Links</li>
+                              
+      <li>
+  
+                          <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+          <span class="none"></span>
+        Apache Software Foundation</a>
+            </li>
+                
+      <li>
+  
+                          <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+          <span class="none"></span>
+        NUS Site</a>
+            </li>
+            </ul>
+                
+                    
+                
+          <hr />
+
+           <div id="poweredBy">
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                                                                                                                   <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+        <img class="builtBy"  alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png"    />
+      </a>
+                      </div>
+          </div>
+        </div>
+        
+                        
+        <div id="bodyColumn"  class="span10" >
+                                  
+            <h1>Code Structure</h1>
+<hr />
+<!-- ### Worker Side
+
+#### Main Classes
+
+<img src="../images/code-structure/main.jpg" style="width: 550px"/>
+
+* **Worker**: start the solver to conduct training or resume from previous training snapshots.
+* **Solver**: construct the neural network and run training algorithms over it. Validation and testing is also done by the solver along the training.
+* **TableDelegate**: delegate for the parameter table physically stored in parameter servers.
+    it runs a thread to communicate with table servers for parameter transferring.
+* **Net**: the neural network consists of multiple layers constructed from input configuration file.
+* **Layer**: the core abstraction, read data (neurons) from connecting layers, and compute the data
+    of itself according to layer specific ComputeFeature functions. Data from the bottom layer is forwarded
+    layer by layer to the top.
+
+#### Data types
+
+<img src="../images/code-structure/layer.jpg" style="width: 700px"/>
+
+* **ComputeFeature**: read data (neurons) from in-coming layers, and compute the data
+    of itself according to layer type. This function can be overrided to implement different
+    types layers.
+* **ComputeGradient**: read gradients (and data) from in-coming layers and compute
+    gradients of parameters and data w.r.t the learning objective (loss).
+
+We adpat the implementation for **PoolingLayer**, **Im2colLayer** and **LRNLayer** from [Caffe](http://caffe.berkeleyvision.org/).
+
+
+<img src="../images/code-structure/darray.jpg" style="width: 400px"/>
+
+* **DArray**: provide the abstraction of distributed array on multiple nodes,
+    supporting array/matrix operations and element-wise operations. Users can use it as a local structure.
+* **LArray**: the local part for the DArray. Each LArray is treated as an
+    independent array, and support all array-related operations.
+* **MemSpace**: manage the memory used by DArray. Distributed memory are allocated
+    and managed by armci. Multiple DArray can share a same MemSpace, the memory
+    will be released when no DArray uses it anymore.
+* **Partition**: maintain both global shape and local partition information.
+    used when two DArray are going to interact.
+* **Shape**: basic class for representing the scope of a DArray/LArray
+* **Range**: basic class for representing the scope of a Partition
+
+### Parameter Server
+
+#### Main classes
+
+<img src="../images/code-structure/uml.jpg" style="width: 750px"/>
+
+* **NetworkService**: provide access to the network (sending and receiving messages). It maintains a queue for received messages, implemented by NetworkQueue.
+* **RequestDispatcher**: pick up next message (request) from the queue, and invoked a method (callback) to process them.
+* **TableServer**: provide access to the data table (parameters). Register callbacks for different types of requests to RequestDispatcher.
+* **GlobalTable**: implement the table. Data is partitioned into multiple Shard objects per table. User-defined consistency model supported by extending TableServerHandler for each table.
+
+#### Data types
+
+<img src="../images/code-structure/type.jpg" style="width: 400px"/>
+
+Table related messages are either of type **RequestBase** which contains different types of request, or of type **TableData** containing a key-value tuple.
+
+#### Control flow and thread model
+
+<img src="../images/code-structure/threads.jpg" alt="uml" style="width: 1000px"/>
+
+The figure above shows how a GET request sent from a worker is processed by the
+table server. The control flow for other types of requests is similar. At
+the server side, there are at least 3 threads running at any time: two by
+NetworkService for sending and receiving message, and at least one by the
+RequestDispatcher for dispatching requests. -->
+                  </div>
+            </div>
+          </div>
+
+    <hr/>
+
+    <footer>
+            <div class="container-fluid">
+                      <div class="row-fluid">
+                                                                          
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+                          </div>
+
+        
+                </div>
+    </footer>
+        </body>
+</html>

Added: websites/staging/singa/trunk/content/docs/kr/communication.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/kr/communication.html (added)
+++ websites/staging/singa/trunk/content/docs/kr/communication.html Wed Jan 13 03:47:56 2016
@@ -0,0 +1,712 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-01-13 
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <meta name="Date-Revision-yyyymmdd" content="20160113" />
+    <meta http-equiv="Content-Language" content="en" />
+    <title>Apache SINGA &#x2013; Communication</title>
+    <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+    <link rel="stylesheet" href="../../css/site.css" />
+    <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+      
+    
+    
+  
+    <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+                          
+        
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+                      
+        
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+                      
+        
+<meta content="Apache SINGA" name="author"/>
+                      
+        
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+          
+                  </head>
+        <body class="topBarEnabled">
+          
+    
+    
+            
+    
+        
+    <a href="https://github.com/apache/incubator-singa">
+      <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+        src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+        alt="Fork me on GitHub">
+    </a>
+  
+                
+                    
+                
+
+    <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+      <div class="navbar-inner">
+                <div class="container-fluid">
+        <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+        </a>
+                
+                                <ul class="nav">
+                          <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../docs/overview.html"  title="Introduction">Introduction</a>
+</li>
+                  
+                      <li>      <a href="../../docs/quick-start.html"  title="Quick Start">Quick Start</a>
+</li>
+                  
+                      <li>      <a href="../../downloads.html"  title="Downloads">Downloads</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li class="dropdown-submenu">
+                                      <a href="../../docs/index.html"  title="Latest">Latest</a>
+              <ul class="dropdown-menu">
+                                  <li>      <a href="../../docs/installation.html"  title="Installation">Installation</a>
+</li>
+                                  <li>      <a href="../../docs/programmer-guide.html"  title="Programmer Guide">Programmer Guide</a>
+</li>
+                                  <li>      <a href="../../docs/distributed-training.html"  title="Distributed Training">Distributed Training</a>
+</li>
+                                  <li>      <a href="../../docs/data.html"  title="Data Preparation">Data Preparation</a>
+</li>
+                                  <li>      <a href="../../docs/checkpoint.html"  title="Checkpoint">Checkpoint</a>
+</li>
+                                  <li>      <a href="../../docs/test.html"  title="Test">Test</a>
+</li>
+                                  <li>      <a href="../../docs/examples.html"  title="Examples">Examples</a>
+</li>
+                                  <li>      <a href="../../docs/debug.html"  title="Debug">Debug</a>
+</li>
+                              </ul>
+            </li>
+                  
+                      <li>      <a href="../../v0.1.0/index.html"  title="v0.1.0">v0.1.0</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../develop/schedule.html"  title="Schedule">Schedule</a>
+</li>
+                  
+                      <li class="dropdown-submenu">
+                                      <a href="../../develop/how-contribute.html"  title="How to Contribute">How to Contribute</a>
+              <ul class="dropdown-menu">
+                                  <li>      <a href="../../develop/contribute-code.html"  title="Code">Code</a>
+</li>
+                                  <li>      <a href="../../develop/contribute-docs.html"  title="Documentation">Documentation</a>
+</li>
+                              </ul>
+            </li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="../../community/source-repository.html"  title="Source Repository">Source Repository</a>
+</li>
+                  
+                      <li>      <a href="../../community/mail-lists.html"  title="Mailing Lists">Mailing Lists</a>
+</li>
+                  
+                      <li>      <a href="../../community/issue-tracking.html"  title="Issue Tracking">Issue Tracking</a>
+</li>
+                  
+                      <li>      <a href="../../community/team-list.html"  title="SINGA Team">SINGA Team</a>
+</li>
+                          </ul>
+      </li>
+                <li class="dropdown">
+        <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+        <ul class="dropdown-menu">
+        
+                      <li>      <a href="http://www.apache.org/"  title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+                  
+                      <li>      <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/"  title="NUS Site">NUS Site</a>
+</li>
+                          </ul>
+      </li>
+                  </ul>
+          
+          
+          
+                   
+                      </div>
+          
+        </div>
+      </div>
+    </div>
+    
+        <div class="container-fluid">
+          <div id="banner">
+        <div class="pull-left">
+                                                  <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+                                                                                                <img src="../../images/singa-logo.png"  alt="Apache SINGA"/>
+                </a>
+                      </div>
+        <div class="pull-right">              <div id="bannerRight">
+                                                                                                <img src="../../images/singa-title.png"  alt="Apache SINGA"/>
+                </div>
+      </div>
+        <div class="clear"><hr/></div>
+      </div>
+
+      <div id="breadcrumbs">
+        <ul class="breadcrumb">
+                
+                    
+                              <li class="">
+                    <a href="../../index.html" title="Apache SINGA">
+        Apache SINGA</a>
+                    <span class="divider">/</span>
+      </li>
+        <li class="active ">Communication</li>
+        
+                
+                    
+      
+                            </ul>
+      </div>
+
+                  
+      <div class="row-fluid">
+        <div id="leftColumn" class="span2">
+          <div class="well sidebar-nav">
+                
+                    
+                <ul class="nav nav-list">
+                    <li class="nav-header">Apache SINGA</li>
+                              
+      <li>
+  
+                          <a href="../../docs/overview.html" title="Introduction">
+          <span class="none"></span>
+        Introduction</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../docs/quick-start.html" title="Quick Start">
+          <span class="none"></span>
+        Quick Start</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../downloads.html" title="Downloads">
+          <span class="none"></span>
+        Downloads</a>
+            </li>
+                              <li class="nav-header">Documentaion</li>
+                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
+      <li>
+  
+                          <a href="../../docs/index.html" title="Latest">
+          <span class="icon-chevron-right"></span>
+        Latest</a>
+                  </li>
+                
+      <li>
+  
+                          <a href="../../v0.1.0/index.html" title="v0.1.0">
+          <span class="none"></span>
+        v0.1.0</a>
+            </li>
+                              <li class="nav-header">Development</li>
+                              
+      <li>
+  
+                          <a href="../../develop/schedule.html" title="Schedule">
+          <span class="none"></span>
+        Schedule</a>
+            </li>
+                                                                                    
+      <li>
+  
+                          <a href="../../develop/how-contribute.html" title="How to Contribute">
+          <span class="icon-chevron-right"></span>
+        How to Contribute</a>
+                  </li>
+                              <li class="nav-header">Community</li>
+                              
+      <li>
+  
+                          <a href="../../community/source-repository.html" title="Source Repository">
+          <span class="none"></span>
+        Source Repository</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/mail-lists.html" title="Mailing Lists">
+          <span class="none"></span>
+        Mailing Lists</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/issue-tracking.html" title="Issue Tracking">
+          <span class="none"></span>
+        Issue Tracking</a>
+            </li>
+                
+      <li>
+  
+                          <a href="../../community/team-list.html" title="SINGA Team">
+          <span class="none"></span>
+        SINGA Team</a>
+            </li>
+                              <li class="nav-header">External Links</li>
+                              
+      <li>
+  
+                          <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+          <span class="none"></span>
+        Apache Software Foundation</a>
+            </li>
+                
+      <li>
+  
+                          <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+          <span class="none"></span>
+        NUS Site</a>
+            </li>
+            </ul>
+                
+                    
+                
+          <hr />
+
+           <div id="poweredBy">
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                            <div class="clear"></div>
+                                                                                                                   <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+        <img class="builtBy"  alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png"    />
+      </a>
+                      </div>
+          </div>
+        </div>
+        
+                        
+        <div id="bodyColumn"  class="span10" >
+                                  
+            <h1>Communication</h1>
+<hr />
+<p>Different messaging libraries has different benefits and drawbacks. For instance, MPI provides fast message passing between GPUs (using GPUDirect), but does not support fault-tolerance well. On the contrary, systems using ZeroMQ can be fault-tolerant, but does not support GPUDirect. The AllReduce function of MPI is also missing in ZeroMQ which is efficient for data aggregation for distributed training. In Singa, we provide general messaging APIs for communication between threads within a process and across processes, and let users choose the underlying implementation (MPI or ZeroMQ) that meets their requirements.</p>
+<p>Singa&#x2019;s messaging library consists of two components, namely the message, and the socket to send and receive messages. <b>Socket</b> refers to a Singa defined data structure instead of the Linux Socket. We will introduce the two components in detail with the following figure as an example architecture.</p>
+<p><img src="../images/arch/arch2.png" style="width: 550px" alt="" /> <img src="../images/arch/comm.png" style="width: 550px" alt="" /> 
+<p><b> Fig.1 - Example physical architecture and network connection</b></p>
+<p>Fig.1 shows an example physical architecture and its network connection. <a href="architecture.html}">Section-partition server side ParamShard</a> has a detailed description of the architecture. Each process consists of one main thread running the stub and multiple background threads running the worker and server tasks. The stub of the main thread forwards messages among threads . The worker and server tasks are performed by the background threads.</p>
+<div class="section">
+<h2><a name="Message"></a>Message</h2>
+<p><object type="image/svg+xml" style="width: 100px" data="../images/msg.svg"> Not supported </object> 
+<p><b> Fig.2 - Logical message format</b></p>
+<p>Fig.2 shows the logical message format which has two parts, the header and the content. The message header includes the sender&#x2019;s and receiver&#x2019;s IDs, each consisting of the group ID and the worker/server ID within the group. The stub forwards messages by looking up an address table based on the receiver&#x2019;s ID. There are two sets of messages according to the message type defined below.</p>
+
+<ul>
+  
+<li>
+<p>kGet/kPut/kRequest/kSync for messages about parameters</p></li>
+  
+<li>
+<p>kFeaBlob/kGradBlob for messages about transferring feature and gradient  blobs of one layer to its neighboring layer</p></li>
+</ul>
+<p>There is a target ID in the header. If the message body is parameters, the target ID is then the parameter ID. Otherwise the message is related to layer feature or gradient, and the target ID consists of the layer ID and the blob ID of that layer. The message content has multiple frames to store the parameter or feature data.</p>
+<p>The API for the base Msg is:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">/**
+ * Msg used to transfer Param info (gradient or value), feature blob, etc
+ * between workers, stubs and servers.
+ *
+ * Each msg has a source addr and dest addr identified by a unique integer.
+ * It is also associated with a target field (value and version) for ease of
+ * getting some meta info (e.g., parameter id) from the msg.
+ *
+ * Other data is added into the message as frames.
+ */
+class Msg {
+ public:
+  ~Msg();
+  Msg();
+  /**
+   * Construct the msg providing source and destination addr.
+   */
+  Msg(int src, int dst);
+  /**
+   * Copy constructor.
+   */
+  Msg(const Msg&amp; msg);
+  /**
+   * Swap the src/dst addr
+   */
+  void SwapAddr();
+  /**
+   * Add a frame (a chunk of bytes) into the message
+   */
+  void AddFrame(const void* addr, int nBytes);
+  /**
+   * @return num of bytes of the current frame.
+   */
+  int FrameSize();
+  /**
+   * @return the pointer to the current frame data.
+   */
+  void* FrameData();
+  /**
+   * @return the data of the current frame as c string
+   */
+  char* FrameStr();
+  /**
+   * Move the cursor to the first frame.
+   */
+  void FirstFrame();
+  /**
+   * Move the cursor to the last frame.
+   */
+  void LastFrame();
+  /**
+   * Move the cursor to the next frame
+   * @return true if the next frame is not NULL; otherwise false
+   */
+  bool NextFrame();
+  /**
+   *  Add a 'format' frame to the msg (like CZMQ's zsock_send).
+   *
+   *  The format is a string that defines the type of each field.
+   *  The format can contain any of these characters, each corresponding to
+   *  one or two arguments:
+   *  i = int (signed)
+   *  1 = uint8_t
+   *  2 = uint16_t
+   *  4 = uint32_t
+   *  8 = uint64_t
+   *  p = void * (sends the pointer value, only meaningful over inproc)
+   *  s = char**
+   *
+   *  Returns size of the added content.
+   */
+  int AddFormatFrame(const char *format, ...);
+  /**
+   *  Parse the current frame added using AddFormatFrame(const char*, ...).
+   *
+   *  The format is a string that defines the type of each field.
+   *  The format can contain any of these characters, each corresponding to
+   *  one or two arguments:
+   *  i = int (signed)
+   *  1 = uint8_t
+   *  2 = uint16_t
+   *  4 = uint32_t
+   *  8 = uint64_t
+   *  p = void * (sends the pointer value, only meaningful over inproc)
+   *  s = char**
+   *
+   *  Returns size of the parsed content.
+   */
+  int ParseFormatFrame(const char* format, ...);
+
+#ifdef USE_ZMQ
+  void ParseFromZmsg(zmsg_t* msg);
+  zmsg_t* DumpToZmsg();
+#endif
+
+  /**
+   * @return msg size in terms of bytes, ignore meta info.
+   */
+  int size() const;
+  /**
+   * Set source addr.
+   * @param addr unique identify one worker/server/stub in the current job
+   */
+  void set_src(int addr) { src_ = addr; }
+  /**
+   * @return source addr.
+   */
+  int src() const { return src_; }
+  /**
+   * Set destination addr.
+   * @param addr unique identify one worker/server/stub in the current job
+   */
+  void set_dst(int addr) { dst_ = addr; }
+  /**
+   * @return dst addr.
+   */
+  int dst() const { return dst_; }
+  /**
+   * Set msg type, e.g., kPut, kGet, kUpdate, kRequest
+   */
+  void set_type(int type) { type_ = type; }
+  /**
+   * @return msg type.
+   */
+  int type() const { return type_; }
+  /**
+   * Set msg target.
+   *
+   * One msg has a target to identify some entity in worker/server/stub.
+   * The target is associated with a version, e.g., Param version.
+   */
+  void set_trgt(int val, int version) {
+    trgt_val_ = val;
+    trgt_version_ = version;
+  }
+  int trgt_val() const {
+    return trgt_val_;
+  }
+  int trgt_version() const {
+    return trgt_version_;
+  }
+
+};
+</pre></div></div>
+<p>In order for a Msg object to be routed, the source and dest address should be attached. This is achieved by calling the set_src and set_dst methods of the Msg object. The address parameter passed to these two methods can be manipulated via a set of helper functions, shown as below.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">/**
+ * Wrapper to generate message address
+ * @param grp worker/server group id
+ * @param id_or_proc worker/server id or procs id
+ * @param type msg type
+ */
+inline int Addr(int grp, int id_or_proc, int type) {
+  return (grp &lt;&lt; 16) | (id_or_proc &lt;&lt; 8) | type;
+}
+
+/**
+ * Parse group id from addr.
+ *
+ * @return group id
+ */
+inline int AddrGrp(int addr) {
+  return addr &gt;&gt; 16;
+}
+/**
+ * Parse worker/server id from addr.
+ *
+ * @return id
+ */
+inline int AddrID(int addr) {
+  static const int mask = (1 &lt;&lt; 8) - 1;
+  return (addr &gt;&gt; 8) &amp; mask;
+}
+
+/**
+ * Parse worker/server procs from addr.
+ *
+ * @return procs id
+ */
+inline int AddrProc(int addr) {
+  return AddrID(addr);
+}
+/**
+ * Parse msg type from addr
+ * @return msg type
+ */
+inline int AddrType(int addr) {
+  static const int mask = (1 &lt;&lt; 8) -1;
+  return addr &amp; mask;
+}
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Socket"></a>Socket</h2>
+<p>In SINGA, there are two types of sockets, the Dealer Socket and the Router Socket, whose names are adapted from ZeroMQ. All connections are of the same type, i.e., Dealer&lt;&#x2013;&gt;Router. The communication between dealers and routers are asynchronous. In other words, one Dealer socket can talk with multiple Router sockets, and one Router socket can talk with multiple Dealer sockets.</p>
+<div class="section">
+<h3><a name="Base_Socket"></a>Base Socket</h3>
+<p>The basic functions of a Singa Socket is to send and receive messages. The APIs are:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">class SocketInterface {
+ public:
+  virtual ~SocketInterface() {}
+  /**
+    * Send a message to connected socket(s), non-blocking. The message
+    * will be deallocated after sending, thus should not be used after
+    * calling Send();
+    *
+    * @param msg The message to be sent
+    * @return 1 for success queuing the message for sending, 0 for failure
+    */
+  virtual int Send(Msg** msg) = 0;
+  /**
+    * Receive a message from any connected socket.
+    *
+    * @return a message pointer if success; nullptr if failure
+    */
+  virtual Msg* Receive() = 0;
+  /**
+   * @return Identifier of the implementation dependent socket. E.g., zsock_t*
+   * for ZeroMQ implementation and rank for MPI implementation.
+   */
+  virtual void* InternalID() const = 0;
+};
+</pre></div></div>
+<p>A poller class is provided to enable asynchronous communication between routers and dealers. One can register a set of SocketInterface objects with a poller instance via calling its Add method, and then call the Wait method of this poll object to wait for the registered SocketInterface objects to be ready for sending and receiving messages. The APIs of the poller class is shown below.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">class Poller {
+ public:
+  Poller();
+  Poller(SocketInterface* socket);
+  /**
+    * Add a socket for polling; Multiple sockets can be polled together by
+    * adding them into the same poller.
+    */
+  void Add(SocketInterface* socket);
+  /**
+    * Poll for all sockets added into this poller.
+    * @param timeout Stop after this number of mseconds
+    * @return pointer To the socket if it has one message in the receiving
+    * queue; nullptr if no message in any sockets,
+    */
+  SocketInterface* Wait(int duration);
+
+  /**
+   * @return true if the poller is terminated due to process interupt
+   */
+  virtual bool Terminated();
+};
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Dealer_Socket"></a>Dealer Socket</h3>
+<p>The Dealer socket inherits from the base Socket. In Singa, every Dealer socket only connects to one Router socket as shown in Fig.1. The connection is set up by connecting the Dealer socket to the endpoint of a Router socket.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">class Dealer : public SocketInterface {
+ public:
+  /*
+   * @param id Local dealer ID within a procs if the dealer is from worker or
+   * server thread, starts from 1 (0 is used by the router); or the connected
+   * remote procs ID for inter-process dealers from the stub thread.
+   */
+  Dealer();
+  explicit Dealer(int id);
+  ~Dealer() override;
+  /**
+    * Setup the connection with the router.
+    *
+    * @param endpoint Identifier of the router. For intra-process
+    * connection, the endpoint follows the format of ZeroMQ, i.e.,
+    * starting with &quot;inproc://&quot;; in Singa, since each process has one
+    * router, hence we can fix the endpoint to be &quot;inproc://router&quot; for
+    * intra-process. For inter-process, the endpoint follows ZeroMQ's
+    * format, i.e., IP:port, where IP is the connected process.
+    * @return 1 connection sets up successfully; 0 otherwise
+    */
+  int Connect(const std::string&amp; endpoint);
+  int Send(Msg** msg) override;
+  Msg* Receive() override;
+  void* InternalID() const override;
+};
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Router_Socket"></a>Router Socket</h3>
+<p>The Router socket inherits from the base Socket. One Router socket connects to at least one Dealer socket. Upon receiving a message, the router forwards it to the appropriate dealer according to the receiver&#x2019;s ID of this message.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">class Router : public SocketInterface {
+ public:
+  Router();
+  /**
+   * There is only one router per procs, hence its local id is 0 and is not set
+   * explicitly.
+   *
+   * @param bufsize Buffer at most this number of messages
+   */
+  explicit Router(int bufsize);
+  ~Router() override;
+  /**
+   * Setup the connection with dealers.
+   *
+   * It automatically binds to the endpoint for intra-process communication,
+   * i.e., &quot;inproc://router&quot;.
+   *
+   * @param endpoint The identifier for the Dealer socket in other process
+   * to connect. It has the format IP:Port, where IP is the host machine.
+   * If endpoint is empty, it means that all connections are
+   * intra-process connection.
+   * @return number of connected dealers.
+   */
+  int Bind(const std::string&amp; endpoint);
+  /**
+   * If the destination socket has not connected yet, buffer this the message.
+   */
+  int Send(Msg** msg) override;
+  Msg* Receive() override;
+  void* InternalID() const override;
+
+};
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="Implementation"></a>Implementation</h2>
+<div class="section">
+<h3><a name="ZeroMQ"></a>ZeroMQ</h3>
+<p><b>Why <a class="externalLink" href="http://zeromq.org/">ZeroMQ</a>?</b> Our previous design used MPI for communication between Singa processes. But MPI is a poor choice when it comes to fault-tolerance, because failure at one node brings down the entire MPI cluster. ZeroMQ, on the other hand, is fault tolerant in the sense that one node failure does not affect the other nodes. ZeroMQ consists of several basic communication patterns that can be easily combined to create more complex network topologies.</p>
+<p><img src="../images/msg-flow.png" style="width: 550px" alt="" /> 
+<p><b> Fig.3 - Messages flow for ZeroMQ</b></p>
+<p>The communication APIs of Singa are similar to the DEALER-ROUTER pattern of ZeroMQ. Hence we can easily implement the Dealer socket using ZeroMQ&#x2019;s DEALER socket, and Router socket using ZeroMQ&#x2019;s ROUTER socket. The intra-process can be implemented using ZeroMQ&#x2019;s inproc transport, and the inter-process can be implemented using the tcp transport (To exploit the Infiniband, we can use the sdp transport). Fig.3 shows the message flow using ZeroMQ as the underlying implementation. The messages sent from dealers has two frames for the message header, and one or more frames for the message content. The messages sent from routers have another frame for the identifier of the destination dealer.</p>
+<p>Besides the DEALER-ROUTER pattern, we may also implement the Dealer socket and Router socket using other ZeroMQ patterns. To be continued.</p></div>
+<div class="section">
+<h3><a name="MPI"></a>MPI</h3>
+<p>Since MPI does not provide intra-process communication, we have to implement it inside the Router and Dealer socket. A simple solution is to allocate one message queue for each socket. Messages sent to one socket is inserted into the queue of that socket. We create a SafeQueue class to ensure the consistency of the queue. All queues are created by the main thread and passed to all sockets&#x2019; constructor via <i>args</i>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">/**
+ * A thread safe queue class.
+ * There would be multiple threads pushing messages into
+ * the queue and only one thread reading and popping the queue.
+ */
+class SafeQueue{
+ public:
+  void Push(Msg* msg);
+  Msg* Front();
+  void Pop();
+  bool empty();
+};
+</pre></div></div>
+<p>For inter-process communication, we serialize the message and call MPI&#x2019;s send/receive functions to transfer them. All inter-process connections are setup by MPI at the beginning. Consequently, the Connect and Bind functions do nothing for both inter-process and intra-process communication.</p>
+<p>MPI&#x2019;s AllReduce function is efficient for data aggregation in distributed training. For example, <a class="externalLink" href="http://arxiv.org/abs/1501.02876">DeepImage of Baidu</a> uses AllReduce to aggregate the updates of parameter from all workers. It has similar architecture as <a href="architecture.html">Fig.2</a>, where every process has a server group and is connected with all other processes. Hence, we can implement DeepImage in Singa by simply using MPI&#x2019;s AllReduce function for inter-process communication.</p></div></div>
+                  </div>
+            </div>
+          </div>
+
+    <hr/>
+
+    <footer>
+            <div class="container-fluid">
+                      <div class="row-fluid">
+                                                                          
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+                          </div>
+
+        
+                </div>
+    </footer>
+        </body>
+</html>




Mime
View raw message