wangwei created SINGA58:

Summary: Fix fanin dimension of weight matrix
Key: SINGA58
URL: https://issues.apache.org/jira/browse/SINGA58
Project: Singa
Issue Type: Improvement
Reporter: wangwei
There are many layers using weight matrix, e.g., convolution layer and innerproduct layer.
Some initialization methods of the weight matrix are based on the fanin of the matrix, i.e.,
the dimension interacts with the other operator. E.g., for v*W, W's fanin is the number of
rows; For v*W^T, W's fanin is the number of columns.
However, the Param::InitValues() method does not know the position (and transposition) of
the matrix in the multiplication operation. We have to fix the fanin dimension. Particularly,
we fix it to the second dimension (i.e., number of columns) and assume users use the weight
matrix correctly.
The current implementation of convolution layer and innerproduct layer have different fanin
dimension, which should be unified to the second dimension. For the convolution layer, its
operation is W*v. W's fanin is already the number of columns. For the innerproduct layer,
the current operation is v*W, which should be updated to v*W^T, then the fanin is the number
of columns.

This message was sent by Atlassian JIRA
(v6.3.4#6332)
