mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephanie Yuan <stephanieyuan1...@gmail.com>
Subject Design Proposal: SVRG Optimization in MXNet Python Module
Date Mon, 20 Aug 2018 22:39:22 GMT
Hi MXNet dev community,

My name is Stephanie Yuan and it's great to join the MXNet dev family!  I'm
proposing a new design doc for implementing SVRG optimization technique in
MXNet Python Module.

*Problem Description: *
SVRG optimization is a technique that complements SGD, which was first
proposed in the paper  Accelerating Stochastic Gradient Descent using
Predicative Variance Reduction
<https://papers.nips.cc/paper/4937-accelerating-stochastic-gradient-descent-using-predictive-variance-reduction.pdf>
in
2013.  It has provable guarantees for strongly convex functions and
converges much faster than SGD. An initial set of experiments using
YearPredictionMSD dataset has been conducted and yields promising results,
which is one of the motivations for this proposal.

*Expected Deliverables:*
The goal is to implement a MXNet Python Module that implements SVRG
optimization technique.

Detailed implementation approaches and Benchmark results can be found in
the Confluence design doc
<https://cwiki.apache.org/confluence/display/MXNET/SVRG+Optimization+in+MXNet+Python+Module>
.

Please let me know if you have any questions! Thank you very much for your
time and considerations!

Cheers,
Stephanie Yuan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message