singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (SINGA-381) Update the autograd API to yeild the gradients
Date Wed, 11 Jul 2018 08:30:00 GMT


ASF subversion and git services commented on SINGA-381:

Commit 81908a82f4c9ea01b1359ed3d8fb4118a5bfd147 in incubator-singa's branch refs/heads/master
from Wang Wei
[;h=81908a8 ]

SINGA-381 - Update the autograd API to yeild the gradients

yield gradients by backward() in; this saves memory by releasing gradients early

> Update the autograd API to yeild the gradients
> ----------------------------------------------
>                 Key: SINGA-381
>                 URL:
>             Project: Singa
>          Issue Type: Improvement
>            Reporter: wangwei
>            Priority: Major
> This ticket is going to update one API in
> {code:java}
> def backward(y, dy=None)
> # returns the gradient tensor one by one using yield
> def gradients(y, dy=None)
> # returns a dictionary: param tensor -> gradient tensor{code}
> With the backward() API, we can update the param immediately after its gradient is available.
Then, the gradient tensor can be deleted and the memory is released.
> The gradients() API keeps all gradient tensors in the memory, which incurs memory overhead.

This message was sent by Atlassian JIRA

View raw message