Return-Path: X-Original-To: apmail-singa-commits-archive@minotaur.apache.org Delivered-To: apmail-singa-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 29E8618165 for ; Wed, 13 May 2015 04:27:37 +0000 (UTC) Received: (qmail 33936 invoked by uid 500); 13 May 2015 04:27:37 -0000 Delivered-To: apmail-singa-commits-archive@singa.apache.org Received: (qmail 33918 invoked by uid 500); 13 May 2015 04:27:37 -0000 Mailing-List: contact commits-help@singa.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@singa.incubator.apache.org Delivered-To: mailing list commits@singa.incubator.apache.org Received: (qmail 33909 invoked by uid 99); 13 May 2015 04:27:37 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 May 2015 04:27:37 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 860D0C0DF2 for ; Wed, 13 May 2015 04:27:36 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.771 X-Spam-Level: * X-Spam-Status: No, score=1.771 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id qi8UxAnGFWce for ; Wed, 13 May 2015 04:27:29 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with SMTP id BFDCE20D8A for ; Wed, 13 May 2015 04:27:28 +0000 (UTC) Received: (qmail 33902 invoked by uid 99); 13 May 2015 04:27:28 -0000 Received: from eris.apache.org (HELO hades.apache.org) (140.211.11.105) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 May 2015 04:27:28 +0000 Received: from hades.apache.org (localhost [127.0.0.1]) by hades.apache.org (ASF Mail Server at hades.apache.org) with ESMTP id 61872AC09F8 for ; Wed, 13 May 2015 04:27:28 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: svn commit: r951118 - in /websites/staging/singa/trunk/content: ./ css/ docs/ Date: Wed, 13 May 2015 04:27:28 -0000 To: commits@singa.incubator.apache.org From: buildbot@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20150513042728.61872AC09F8@hades.apache.org> Author: buildbot Date: Wed May 13 04:27:27 2015 New Revision: 951118 Log: Staging update by buildbot for singa Modified: websites/staging/singa/trunk/content/ (props changed) websites/staging/singa/trunk/content/community.html websites/staging/singa/trunk/content/css/site.css websites/staging/singa/trunk/content/docs.html websites/staging/singa/trunk/content/docs/architecture.html websites/staging/singa/trunk/content/docs/code-structure.html websites/staging/singa/trunk/content/docs/communication.html websites/staging/singa/trunk/content/docs/installation.html websites/staging/singa/trunk/content/docs/neuralnet-partition.html websites/staging/singa/trunk/content/docs/programming-model.html websites/staging/singa/trunk/content/index.html websites/staging/singa/trunk/content/introduction.html websites/staging/singa/trunk/content/quick-start.html Propchange: websites/staging/singa/trunk/content/ ------------------------------------------------------------------------------ --- cms:source-revision (original) +++ cms:source-revision Wed May 13 04:27:27 2015 @@ -1 +1 @@ -1678942 +1679136 Modified: websites/staging/singa/trunk/content/community.html ============================================================================== --- websites/staging/singa/trunk/content/community.html (original) +++ websites/staging/singa/trunk/content/community.html Wed May 13 04:27:27 2015 @@ -1,13 +1,13 @@ - + Apache SINGA – Community @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -208,7 +213,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -228,7 +234,7 @@
      - Apache Incubator + Apache Incubator @@ -249,12 +255,9 @@
      -

      Copyright © 2015 - The Apache Software Foundation. - All rights reserved. - -

      -
      + +

      Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.

      +
      Modified: websites/staging/singa/trunk/content/css/site.css ============================================================================== --- websites/staging/singa/trunk/content/css/site.css (original) +++ websites/staging/singa/trunk/content/css/site.css Wed May 13 04:27:27 2015 @@ -0,0 +1,6 @@ +.centered{ + text-align:center; + margin-top:0px; + margin-bottom:0px; + padding:0px; +} Modified: websites/staging/singa/trunk/content/docs.html ============================================================================== --- websites/staging/singa/trunk/content/docs.html (original) +++ websites/staging/singa/trunk/content/docs.html Wed May 13 04:27:27 2015 @@ -1,13 +1,13 @@ - + Apache SINGA – Documentation @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -208,7 +213,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -228,7 +234,7 @@
      - Apache Incubator + Apache Incubator @@ -264,12 +270,9 @@
      -

      Copyright © 2015 - The Apache Software Foundation. - All rights reserved. - -

      -
      + +

      Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.

      +
      Modified: websites/staging/singa/trunk/content/docs/architecture.html ============================================================================== --- websites/staging/singa/trunk/content/docs/architecture.html (original) +++ websites/staging/singa/trunk/content/docs/architecture.html Wed May 13 04:27:27 2015 @@ -1,13 +1,13 @@ - + Apache SINGA – System Architecture @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -206,7 +211,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -226,7 +232,7 @@
      - Apache Incubator + Apache Incubator @@ -309,12 +315,9 @@
      -

      Copyright © 2015 - The Apache Software Foundation. - All rights reserved. - -

      -
      + +

      Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.

      +
      Modified: websites/staging/singa/trunk/content/docs/code-structure.html ============================================================================== --- websites/staging/singa/trunk/content/docs/code-structure.html (original) +++ websites/staging/singa/trunk/content/docs/code-structure.html Wed May 13 04:27:27 2015 @@ -1,13 +1,13 @@ - + Apache SINGA – Code Structure @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -206,7 +211,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -226,7 +232,7 @@
      - Apache Incubator + Apache Incubator @@ -242,7 +248,7 @@

      Worker Side

      Main Classes

      -

      +

        @@ -258,7 +264,7 @@

      Data types

      -

      +

        @@ -267,7 +273,7 @@
      • ComputeGradient: read gradients (and data) from in-coming layers and compute gradients of parameters and data w.r.t the learning objective (loss).

      We adpat the implementation for PoolingLayer, Im2colLayer and LRNLayer from Caffe.

      -

      +

        @@ -287,7 +293,7 @@

        Parameter Server

        Main classes

        -

        +

          @@ -301,11 +307,11 @@

        Data types

        -

        +

        Table related messages are either of type RequestBase which contains different types of request, or of type TableData containing a key-value tuple.

        Control flow and thread model

        -

        uml

        +

        uml

        The figure above shows how a GET request sent from a worker is processed by the table server. The control flow for other types of requests is similar. At the server side, there are at least 3 threads running at any time: two by NetworkService for sending and receiving message, and at least one by the RequestDispatcher for dispatching requests.

      @@ -316,12 +322,9 @@
      -

      Copyright © 2015 - The Apache Software Foundation. - All rights reserved. - -

      -
      + +

      Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.

      +
      Modified: websites/staging/singa/trunk/content/docs/communication.html ============================================================================== --- websites/staging/singa/trunk/content/docs/communication.html (original) +++ websites/staging/singa/trunk/content/docs/communication.html Wed May 13 04:27:27 2015 @@ -1,13 +1,13 @@ - + Apache SINGA – Communication @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -206,7 +211,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -226,7 +232,7 @@
      - Apache Incubator + Apache Incubator @@ -513,12 +519,9 @@ All messages in SINGA are of multi-frame
      -

      Copyright © 2015 - The Apache Software Foundation. - All rights reserved. - -

      -
      + +

      Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.

      +
      Modified: websites/staging/singa/trunk/content/docs/installation.html ============================================================================== --- websites/staging/singa/trunk/content/docs/installation.html (original) +++ websites/staging/singa/trunk/content/docs/installation.html Wed May 13 04:27:27 2015 @@ -1,13 +1,13 @@ - + Apache SINGA – Installation @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -206,7 +211,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -226,7 +232,7 @@
      - Apache Incubator + Apache Incubator @@ -357,12 +363,9 @@ zeromq zeromq lib
      -

      Copyright © 2015 - The Apache Software Foundation. - All rights reserved. - -

      -
      + +

      Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.

      +
      Modified: websites/staging/singa/trunk/content/docs/neuralnet-partition.html ============================================================================== --- websites/staging/singa/trunk/content/docs/neuralnet-partition.html (original) +++ websites/staging/singa/trunk/content/docs/neuralnet-partition.html Wed May 13 04:27:27 2015 @@ -1,13 +1,13 @@ - + Apache SINGA – Neural Network Partition @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -206,7 +211,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -226,7 +232,7 @@
      - Apache Incubator + Apache Incubator @@ -240,11 +246,11 @@

      The purposes of partitioning neural network is to distribute the partitions onto different working units (e.g., threads or nodes, called workers in this article) and parallelize the processing. Another reason for partition is to handle large neural network which cannot be hold in a single node. For instance, to train models against images with high resolution we need large neural networks (in terms of training parameters).

      Since Layer is the first class citizen in SIGNA, we do the partition against layers. Specifically, we support partitions at two levels. First, users can configure the location (i.e., worker ID) of each layer. In this way, users assign one worker for each layer. Secondly, for one layer, we can partition its neurons or partition the instances (e.g, images). They are called layer partition and data partition respectively. We illustrate the two types of partitions using an simple convolutional neural network.

      -

      +

      The above figure shows a convolutional neural network without any partition. It has 8 layers in total (one rectangular represents one layer). The first layer is DataLayer (data) which reads data from local disk files/databases (or HDFS). The second layer is a MnistLayer which parses the records from MNIST data to get the pixels of a batch of 28 images (each image is of size 28x28). The LabelLayer (label) parses the records to get the label of each image in the batch. The ConvolutionalLayer (conv1) transforms the input image to the shape of 8x27x27. The ReLULayer (relu1) conducts elementwise transformations. The PoolingLayer (pool1) sub-samples the images. The fc1 layer is fully connected with pool1 layer. It mulitplies each image with a weight matrix to generate a 10 dimension hidden feature which is then normalized by a SoftmaxLossLayer to get the prediction.

      -

      +

      The above figure shows the convolutional neural network after partitioning all layers except the DataLayer and ParserLayers, into 3 partitions using data partition. The read layers process 4 images of the batch, the black and blue layers process 2 images respectively. Some helper layers, i.e., SliceLayer, ConcateLayer, BridgeSrcLayer, BridgeDstLayer and SplitLayer, are added automatically by our partition algorithm. Layers of the same color resident in the same worker. There would be data transferring across different workers at the boundary layers (i.e., BridgeSrcLayer and BridgeDstLayer), e.g., between s-slice-mnist-conv1 and d-slice-mnist-conv1.

      -

      +

      The above figure shows the convolutional neural network after partitioning all layers except the DataLayer and ParserLayers, into 2 partitions using layer partition. We can see that each layer processes all 8 images from the batch. But different partitions process different part of one image. For instance, the layer conv1-00 process only 4 channels. The other 4 channels are processed by conv1-01 which residents in another worker.

      Since the partition is done at the layer level, we can apply different partitions for different layers to get a hybrid partition for the whole neural network. Moreover, we can also specify the layer locations to locate different layers to different workers.

      @@ -256,12 +262,9 @@
      -

      Copyright © 2015 - The Apache Software Foundation. - All rights reserved. - -

      -
      + +

      Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.

      +
      Modified: websites/staging/singa/trunk/content/docs/programming-model.html ============================================================================== --- websites/staging/singa/trunk/content/docs/programming-model.html (original) +++ websites/staging/singa/trunk/content/docs/programming-model.html Wed May 13 04:27:27 2015 @@ -1,13 +1,13 @@ - + Apache SINGA – Programming Model @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -206,7 +211,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -226,7 +232,7 @@
      - Apache Incubator + Apache Incubator @@ -345,12 +351,9 @@ LayerProto{
      -

      Copyright © 2015 - The Apache Software Foundation. - All rights reserved. - -

      -
      + +

      Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.

      +
      Modified: websites/staging/singa/trunk/content/index.html ============================================================================== --- websites/staging/singa/trunk/content/index.html (original) +++ websites/staging/singa/trunk/content/index.html Wed May 13 04:27:27 2015 @@ -1,15 +1,15 @@ - + - Apache SINGA – Welcome to SINGA + Apache SINGA – Welcome to Apache SINGA @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -128,7 +133,7 @@ Apache SINGA / -
    • Welcome to SINGA
      • +
    • Welcome to Apache SINGA
      • @@ -206,7 +211,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -226,7 +232,7 @@
      - Apache Incubator + Apache Incubator @@ -236,7 +242,7 @@
      -

      Welcome to SINGA

      +

      Welcome to Apache SINGA

      Quick Start

      @@ -283,12 +289,9 @@
      -

      Copyright © 2015 - The Apache Software Foundation. - All rights reserved. - -

      -
      + +

      Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.

      +
      Modified: websites/staging/singa/trunk/content/introduction.html ============================================================================== --- websites/staging/singa/trunk/content/introduction.html (original) +++ websites/staging/singa/trunk/content/introduction.html Wed May 13 04:27:27 2015 @@ -1,13 +1,13 @@ - + Apache SINGA – Introduction @@ -87,6 +87,11 @@
  • + Community +
      +
    +
    • +
  • External Links
      @@ -206,7 +211,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -226,7 +232,7 @@
      - Apache Incubator + Apache Incubator @@ -275,7 +281,7 @@

      Considering extensibility, we make our core data structures (e.g., Layer) and operations general enough for programmers to override.

      System Overview

      -

      SINGA software stack

      +

      SINGA software stack

      Three goals are considered in designing SINGA, namely ease of use, scalability and extensibility. We will introduce them together with the software stack as shown in the above figure. Algorithms for deep learning models are complex to code and hard to train. To make it ease of use, we provide a simple concept ‘Layer’ to construct deep complex models. Built-in Layer implementations include common layers, e.g., convolution layer and fully connected layer. Users can configure their models by combining these built-in layers through web interface or configuration files. Once the model and training data is configured, we start SINGA to conduct the training using the standard training algorithm (Back-Propagation,BP or Contrastive Divergence, CD) on a cluster of nodes and visualize the training performance to users (e.g., through web interface). Advanced users can also implement their own layers by overloading the base Layer class through Python, Matlab, etc wrappers. Distr ibutedArray is proposed for easy array operations that are heavily used for realizing layer logics. SINGA manages the distributed arrays (stored across multiple nodes) automatically and efficiently based on MPI. Training scalability is achieved by partitioning the training data and model onto multiple computing nodes and parallelizing the computation. A logically centralized parameter server maintains the model parameters in a ParameterTable. Computing nodes work according to the consistency policy and send information to the parameter server which updates the parameters based on SGD (stochastic gradient descent) algorithms. Besides the Layer class, other components like SGD algorithms and consistency module are also extensible. - + Apache SINGA – Quick Start @@ -87,6 +87,11 @@

  • + Community +
      +
    +
    • +
  • External Links
      @@ -206,7 +211,8 @@ Programming Model -
    • External Links
      • +
    • Community
      • +
    • External Links
    • @@ -226,7 +232,7 @@
      - Apache Incubator + Apache Incubator @@ -311,12 +317,9 @@ cd ..