singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SINGA-381) Update the autograd API to yeild the gradients
Date Wed, 11 Jul 2018 08:30:00 GMT

    [ https://issues.apache.org/jira/browse/SINGA-381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539716#comment-16539716
] 

ASF subversion and git services commented on SINGA-381:
-------------------------------------------------------

Commit 81908a82f4c9ea01b1359ed3d8fb4118a5bfd147 in incubator-singa's branch refs/heads/master
from Wang Wei
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=81908a8 ]

SINGA-381 - Update the autograd API to yeild the gradients

yield gradients by backward() in autograd.py; this saves memory by releasing gradients early


> Update the autograd API to yeild the gradients
> ----------------------------------------------
>
>                 Key: SINGA-381
>                 URL: https://issues.apache.org/jira/browse/SINGA-381
>             Project: Singa
>          Issue Type: Improvement
>            Reporter: wangwei
>            Priority: Major
>
> This ticket is going to update one API in autograd.py
> {code:java}
> def backward(y, dy=None)
> # returns the gradient tensor one by one using yield
> def gradients(y, dy=None)
> # returns a dictionary: param tensor -> gradient tensor{code}
>  
> With the backward() API, we can update the param immediately after its gradient is available.
Then, the gradient tensor can be deleted and the memory is released.
>  
> The gradients() API keeps all gradient tensors in the memory, which incurs memory overhead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message