From commits-return-108329-archive-asf-public=cust-asf.ponee.io@mxnet.incubator.apache.org Tue Dec 3 18:51:04 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id A0F0018065B for ; Tue, 3 Dec 2019 19:51:03 +0100 (CET) Received: (qmail 49941 invoked by uid 500); 3 Dec 2019 18:51:02 -0000 Mailing-List: contact commits-help@mxnet.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@mxnet.incubator.apache.org Delivered-To: mailing list commits@mxnet.incubator.apache.org Received: (qmail 49862 invoked by uid 99); 3 Dec 2019 18:51:01 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Dec 2019 18:51:01 +0000 From: GitBox To: commits@mxnet.apache.org Subject: [GitHub] [incubator-mxnet] MoisesHer commented on issue #16893: Multi-tensor LAMB Message-ID: <157539906186.6155.7028587780331235354.gitbox@gitbox.apache.org> Date: Tue, 03 Dec 2019 18:51:01 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit MoisesHer commented on issue #16893: Multi-tensor LAMB URL: https://github.com/apache/incubator-mxnet/pull/16893#issuecomment-561304820 > Ping @MoisesHer Is the MultiLAMB optimizer a generalization of LAMB optimizer? If so, why do we keep the LAMB optimizer? If not, please add documentation or a reference to the docstring. Thank you! Sorry, since we were having different numbers of states I was not sure how to reuse the previous optimizer. After Haibin suggestion, now we have same numbers of states. I have merged both optimizers. If aggregation number is <=1 or gradients/weights is a unique NDArray, the single-tensor implementation is used, otherwise, the multi-tensor implementation is used. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services