singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wangwei (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SINGA-19) Slice large Param objects for load-balance
Date Fri, 19 Jun 2015 06:29:00 GMT
wangwei created SINGA-19:
----------------------------

             Summary: Slice large Param objects for load-balance
                 Key: SINGA-19
                 URL: https://issues.apache.org/jira/browse/SINGA-19
             Project: Singa
          Issue Type: New Feature
            Reporter: wangwei
            Assignee: wangwei


Some Param objects in deep learning model are much larger than other Param objects. For example,
the weight matrix is usually 100 times larger than the bias vector. The difference in Param
size causes two problems,

1. if there are multiple servers in one server group, then the servers may be assigned different
number of parameters to update.
2. if there are multiple server groups, e.g., in distributed Hogwild framework, then these
server groups may be assigned different number of parameters to maintain.

This ticket its to slice large Param objects to solve the load-balance problem. The slicing
operations are done in the stub thread to make them transparent to both workers and servers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message