singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SINGA-37) Enable users to set parameter sharing in model configuration
Date Tue, 21 Jul 2015 14:07:04 GMT

    [ https://issues.apache.org/jira/browse/SINGA-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635161#comment-14635161
] 

ASF subversion and git services commented on SINGA-37:
------------------------------------------------------

Commit 4502f41cd0b4c84803d398972549f506e7ac6aa4 in incubator-singa's branch refs/heads/master
from wang wei
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=4502f41 ]

SINGA-37 Enable users to set parameter sharing in model configuration

A share_from field is added into the ParamProto.
It indicates the name of the Param that owns the values of this param.
The NeuralNet class will then share the data_ Blob from the owner Param.
In specific, it will consider share_from firstly and then consider param sharing
due to model partition (on dim 0).

Added a header file tinydir.h for directory management, e.g., list files under the directory

Copy param config for share_from; users do not need to config a param (e.g., init_method)
if this param has a share_from field; NeuralNet will copy the config from share_from proto.

Fixbug from copying param proto for sharing params, which overwrites the name of the param
because MergeFrom/CopyFrom overwrite singular field.


> Enable users to set parameter sharing in model configuration
> ------------------------------------------------------------
>
>                 Key: SINGA-37
>                 URL: https://issues.apache.org/jira/browse/SINGA-37
>             Project: Singa
>          Issue Type: New Feature
>            Reporter: wangwei
>
> For some models (e.g., RBM, RNN), the parameters of some layers are shared (tied). To
enable this feature, SINGA needs to provide a configuration field which indicates from whom
to share the parameter.
> In specific, we can add one field in the ParamProto:
> {code}
> // the name of a Param who owns the Param and shares its values with this Param.
> optional string share_from;  
> {code} 
> The NeuralNet class with share the Params based on the share_from field. Params only
share values, i.e., data field. Everyone has its own gradient, history, etc fields. SINGA
will automatically average the gradients for the shared Params before sending the update request.
>  
> This ticket will not consider the case in which the original (owner of a shared) Param
object is partitioned due to model partitioning, e.g., the weight matrix is partitioned into
two sub-matrix if the inner-product layer is partitioned on dimension 1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message