singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (SINGA-46) Fix a bug in to scale the gradients
Date Wed, 12 Aug 2015 09:00:52 GMT


ASF subversion and git services commented on SINGA-46:

Commit 538736c4ae6172d2f66fe09d9466ac878e8c8415 in incubator-singa's branch refs/heads/master
from wang sheng
[;h=538736c ]

SINGA-46 Fix a bug in to scale the gradients

merge to master

> Fix a bug in to scale the gradients
> ----------------------------------------------
>                 Key: SINGA-46
>                 URL:
>             Project: Singa
>          Issue Type: Bug
>            Reporter: wangwei
> The Updater's Update function has an argument grad_scale (default value is 1.0), which
is used to scale the gradients of parameters. For instance, when n workers compute over one
mini-batch (each worker is assigned 1/n records), then their gradients should be averaged.
We can do the average by passing grad_scale=1/n.
> Some updaters in, e.g., AdaGradUpdater, forget to scale the gradients by grad_scale.
The bug can be fixed by
> {code}
> if (grad_scale != 1)
>     grad *= grad_scale;
> {code}

This message was sent by Atlassian JIRA

View raw message