Mailing-List: contact commits-help@singa.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@singa.incubator.apache.org
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: svn commit: r1707954 - in
 /incubator/singa/site/trunk/content/markdown/docs:
 index.md test.md
Date: Sun, 11 Oct 2015 07:24:30 -0000
To: commits@singa.incubator.apache.org
From: wangwei@apache.org
Message-Id: <20151011072431.21B0F3A043E@svn01-us-west.apache.org>

Author: wangwei
Date: Sun Oct 11 07:24:30 2015
New Revision: 1707954

URL: http://svn.apache.org/viewvc?rev=1707954&view=rev
Log:
add docs for performance test and feature extraction in test.md

Added:
    incubator/singa/site/trunk/content/markdown/docs/test.md
Modified:
    incubator/singa/site/trunk/content/markdown/docs/index.md

Modified: incubator/singa/site/trunk/content/markdown/docs/index.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/index.md?rev=1707954&r1=1707953&r2=1707954&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/index.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs/index.md Sun Oct 11 07:24:30 2015
@@ -12,6 +12,7 @@
 * [Distributed Training](distributed-training.html)
 * [Data Preparation](data.html)
 * [Checkpoint and Resume](checkpoint.html)
+* [Performance test and Feature extraction](test.html)
 * [Examples](examples.html)
     * Feed-forward models
         * [CNN](cnn.html)

Added: incubator/singa/site/trunk/content/markdown/docs/test.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/test.md?rev=1707954&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/test.md (added)
+++ incubator/singa/site/trunk/content/markdown/docs/test.md Sun Oct 11 07:24:30 2015
@@ -0,0 +1,82 @@
+# Performance Test and Feature Extraction
+
+----
+
+Once SINGA finishes the training of a model, it would checkpoint the model parameters
+into disk files under the [checkpoint folder](checkpoint.html). Model parameters can also be dumped
+into this folder periodically during training if the
+[checkpoint configuration[(checkpoint.html) fields are set. With the checkpoint
+files, we can load the model parameters to conduct performance test or feature extraction
+against new data.
+
+To load the model parameters from checkpoint files, we need to add the paths of
+checkpoint files in the job configuration file
+
+    checkpoint_path: PATH_TO_CHECKPOINT_FILE1
+    checkpoint_path: PATH_TO_CHECKPOINT_FILE2
+    ...
+
+The new dataset is configured by specifying the ``test_step`` and the data input
+layer, e.g. the following configuration is for a dataset with 100*100 instances.
+
+    test_steps: 100
+    net {
+      layer {
+        name: "input"
+        store_conf {
+          backend: "kvfile"
+          path: PATH_TO_TEST_KVFILE
+          batchsize: 100
+        }
+      }
+      ...
+    }
+
+## Performance Test
+
+This application is to test the performance, e.g., accuracy, of the previously
+trained model. Depending on the application, the test data may have ground truth
+labels or not. For example, if the model is trained for image classification,
+the test images must have ground truth labels to calculate the accuracy; if the
+model is an auto-encoder, the performance could be measured by reconstruction error, which
+does not require extra labels. For both cases, there would be a layer that calculates
+the performance, e.g., the `SoftmaxLossLayer`.
+
+The job configuration file for the cifar10 example can be used directly for testing after
+adding the checkpoint path. The running command is
+
+
+    $ ./bin/singa-run.sh -conf examples/cifar10/job.conf -test
+
+The performance would be output on the screen like,
+
+
+    Load from checkpoint file examples/cifar10/checkpoint/step50000-worker0
+    accuracy = 0.728000, loss = 0.807645
+
+## Feature extraction
+
+Since deep learning models are good at learning features, feature extraction for
+is a major functionality of deep learning models, e.g., we can extract features
+from the fully connected layers of [AlexNet](www.cs.toronto.edu/~fritz/absps/imagenet.pdf) as image features for image retrieval.
+To extract the features from one layer, we simply add an output layer after that layer.
+For instance, to extract the fully connected (with name `ip1`) layer of the cifar10 example model,
+we replace the `SoftmaxLossLayer` with a `CSVOutputLayer` which extracts the features into a CSV file,
+
+    layer {
+      name: "ip1"
+    }
+    layer {
+      name: "output"
+      type: kCSVOutput
+      srclayers: "ip1"
+      store_conf {
+        backend: "textfile"
+        path: OUTPUT_FILE_PATH
+      }
+    }
+
+The input layer and test steps, and the running command are the same as in *Performance Test* section.
+
+If the output layer is connected to a layer that predicts labels of images,
+the output layer would then write the prediction results into files.