singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wangwei (JIRA)" <>
Subject [jira] [Created] (SINGA-58) Fix fan-in dimension of weight matrix
Date Sat, 15 Aug 2015 03:02:46 GMT
wangwei created SINGA-58:

             Summary: Fix fan-in dimension of weight matrix
                 Key: SINGA-58
             Project: Singa
          Issue Type: Improvement
            Reporter: wangwei

There are many layers using weight matrix, e.g., convolution layer and innerproduct layer.
Some initialization methods of the weight matrix are based on the fan-in of the matrix, i.e.,
the dimension interacts with the other operator. E.g., for v*W, W's fan-in is the number of
rows; For v*W^T, W's fan-in is the number of columns.

However, the Param::InitValues() method does not know the position (and transposition) of
the matrix in the multiplication operation. We have to fix the fan-in dimension. Particularly,
we fix it to the second dimension (i.e., number of columns) and assume users use the weight
matrix correctly.

The current implementation of convolution layer and innerproduct layer have different fan-in
dimension, which should be unified to the second dimension. For the convolution layer, its
operation is W*v. W's fan-in is already the number of columns. For the innerproduct layer,
the current operation is v*W, which should be updated to v*W^T, then the fan-in is the number
of columns.

This message was sent by Atlassian JIRA

View raw message