flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Rohrmann (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1994) Add stepsize calculation schemes to SGD
Date Fri, 08 May 2015 10:18:03 GMT
Till Rohrmann created FLINK-1994:
------------------------------------

             Summary: Add stepsize calculation schemes to SGD
                 Key: FLINK-1994
                 URL: https://issues.apache.org/jira/browse/FLINK-1994
             Project: Flink
          Issue Type: Improvement
          Components: Machine Learning Library
            Reporter: Till Rohrmann
            Priority: Minor


The current SGD implementation uses as gain for the weight updates the formula {{stepsize/sqrt(iterationNumber)}}.
It would be good to make the gain calculation configurable and to provide different strategies
for that. For example:

* stepsize/(1 + iterationNumber)
* stepsize*(1 + regularization * stepsize * iterationNumber)^(-3/4)

See also how to properly select the gains [1].

Resources:
[1] http://arxiv.org/pdf/1107.2490.pdf




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message