singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (SINGA-58) Fix fan-in dimension of weight matrix
Date Sat, 15 Aug 2015 08:11:46 GMT


ASF subversion and git services commented on SINGA-58:

Commit 3ec12b92636ec893be845b08580ec54e302d6689 in incubator-singa's branch refs/heads/master
from Wei Wang
[;h=3ec12b9 ]

SINGA-55 Refactor and singa.h

A Driver class is implemented for initializing SINGA including parsing job ID,
and registering built-in subclasses of Layer, Updater, Worker and Param.
May use it to init MPI if using MPI as the message passing lib.

The file is updated to provide an example main func.

Update the GaussianSqrtFanin init method for Param (ref SINGA-58).

> Fix fan-in dimension of weight matrix
> -------------------------------------
>                 Key: SINGA-58
>                 URL:
>             Project: Singa
>          Issue Type: Improvement
>            Reporter: wangwei
> There are many layers using weight matrix, e.g., convolution layer and innerproduct layer.
Some initialization methods of the weight matrix are based on the fan-in of the matrix, i.e.,
the dimension interacts with the other operator. E.g., for v*W, W's fan-in is the number of
rows; For v*W^T, W's fan-in is the number of columns.
> However, the Param::InitValues() method does not know the position (and transposition)
of the matrix in the multiplication operation. We have to fix the fan-in dimension. Particularly,
we fix it to the second dimension (i.e., number of columns) and assume users use the weight
matrix correctly.
> The current implementation of convolution layer and innerproduct layer have different
fan-in dimension, which should be unified to the second dimension. For the convolution layer,
its operation is W*v. W's fan-in is already the number of columns. For the innerproduct layer,
the current operation is v*W, which should be updated to v*W^T, then the fan-in is the number
of columns.

This message was sent by Atlassian JIRA

View raw message