singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SINGA-19) Slice large Param objects for load-balance
Date Tue, 23 Jun 2015 14:15:00 GMT

    [ https://issues.apache.org/jira/browse/SINGA-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597697#comment-14597697
] 

ASF subversion and git services commented on SINGA-19:
------------------------------------------------------

Commit 56d32e8a0dbfa3226053442d6b97602c0e386936 in incubator-singa's branch refs/heads/master
from wang wei
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=56d32e8 ]

SINGA-19 Slice large Param objects for load-balance

Fixed a bug from transferring bool variable through Msg.


> Slice large Param objects for load-balance
> ------------------------------------------
>
>                 Key: SINGA-19
>                 URL: https://issues.apache.org/jira/browse/SINGA-19
>             Project: Singa
>          Issue Type: New Feature
>            Reporter: wangwei
>            Assignee: wangwei
>
> Some Param objects in deep learning models are much larger than other Param objects.
For example, a weight matrix is usually 100 times larger than a bias vector. The difference
in Param size causes two problems,
> 1. if there are multiple servers in one server group, then the servers may be assigned
different number of parameters to update.
> 2. if there are multiple server groups, e.g., in distributed Hogwild framework, then
these server groups may be assigned different number of parameters to maintain.
> This ticket its to slice large Param objects to solve the load-balance problem. The slicing
operations are done in the stub thread to make them transparent to both workers and servers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message