Return-Path: X-Original-To: apmail-singa-dev-archive@minotaur.apache.org Delivered-To: apmail-singa-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A7FEB18B7F for ; Wed, 24 Feb 2016 09:27:20 +0000 (UTC) Received: (qmail 78832 invoked by uid 500); 24 Feb 2016 09:26:22 -0000 Delivered-To: apmail-singa-dev-archive@singa.apache.org Received: (qmail 78803 invoked by uid 500); 24 Feb 2016 09:26:22 -0000 Mailing-List: contact dev-help@singa.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@singa.incubator.apache.org Delivered-To: mailing list dev@singa.incubator.apache.org Received: (qmail 78792 invoked by uid 99); 24 Feb 2016 09:26:22 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Feb 2016 09:26:22 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id BEB341A000F for ; Wed, 24 Feb 2016 09:26:21 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.549 X-Spam-Level: X-Spam-Status: No, score=-3.549 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.329] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id BiJn-UEV3woP for ; Wed, 24 Feb 2016 09:26:19 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with SMTP id 04FFF5FBC6 for ; Wed, 24 Feb 2016 09:26:18 +0000 (UTC) Received: (qmail 78262 invoked by uid 99); 24 Feb 2016 09:26:18 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Feb 2016 09:26:18 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 2380E2C1F58 for ; Wed, 24 Feb 2016 09:26:18 +0000 (UTC) Date: Wed, 24 Feb 2016 09:26:18 +0000 (UTC) From: "ASF subversion and git services (JIRA)" To: dev@singa.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (SINGA-145) New SGD based optimization Updaters: AdaDelta, Adam, AdamMax MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SINGA-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15162722#comment-15162722 ] ASF subversion and git services commented on SINGA-145: ------------------------------------------------------- Commit 68140079edc5a1c35676a4553e8cc1498ad8784a in incubator-singa's branch refs/heads/master from ijingo [ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=6814007 ] SINGA-145 New SGD based optimization Updaters: AdaDelta, Adam, AdamMax exclude unnecessary low level optimization > New SGD based optimization Updaters: AdaDelta, Adam, AdamMax > ------------------------------------------------------------ > > Key: SINGA-145 > URL: https://issues.apache.org/jira/browse/SINGA-145 > Project: Singa > Issue Type: New Feature > Environment: Universal > Reporter: Wang Ji > Priority: Minor > > This ticket implements three Stochastic Gradient Descent (SGD) based optimization algorithms in Updater.cc, i.e. AdaDelta, Adaptive Moment Estimation(Adam) with its variant AdamMax .These algorithms adapt the learning rate to the parameters, performing larger updates for infrequent and smaller updates for frequent parameters. > For algorithm details, refer to AdaDelata (http://arxiv.org/abs/1212.5701), Adam (http://arxiv.org/pdf/1412.6980.pdf). -- This message was sent by Atlassian JIRA (v6.3.4#6332)