From dev-return-2730-archive-asf-public=cust-asf.ponee.io@mxnet.incubator.apache.org Wed May 2 12:50:31 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id EB5B018065D for ; Wed, 2 May 2018 12:50:30 +0200 (CEST) Received: (qmail 78004 invoked by uid 500); 2 May 2018 10:50:30 -0000 Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@mxnet.incubator.apache.org Delivered-To: mailing list dev@mxnet.incubator.apache.org Received: (qmail 77988 invoked by uid 99); 2 May 2018 10:50:29 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 May 2018 10:50:29 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D9650C7D1E for ; Wed, 2 May 2018 10:50:28 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.869 X-Spam-Level: ** X-Spam-Status: No, score=2.869 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_REPLY=1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 5t6pToi0s9Fn for ; Wed, 2 May 2018 10:50:27 +0000 (UTC) Received: from mail-lf0-f48.google.com (mail-lf0-f48.google.com [209.85.215.48]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 70FED5F58A for ; Wed, 2 May 2018 10:50:26 +0000 (UTC) Received: by mail-lf0-f48.google.com with SMTP id w8-v6so20191758lfe.3 for ; Wed, 02 May 2018 03:50:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=29ZV4GOZT/03Bjf+MyTwKlvuQUYq4U6PK43stGTt/dg=; b=V3pUebZEpwPPiKn9pE4QESj/bKDu2dC0I7KpoQH3C3Iir2kO/HhUVD4wJivEUBejwl OmQiIqXYEG4rpSrYiU1ubFkH5HXRSVeBfDDPcPydRzcZ341chcQxGIrtgio9elsovj/B HXmMzqLtZFQj6O3xQ4JTjoa1hy7OUfyerdZirW7B2wSdd43TLPTclbMI0yMLlZJiMQHN x0WFZzdE4uzV6S6YcN8GG1s35dM6okI8H8JP3GtFmUvv6EDQaGqpAlxXGQSRGPNVyydO OFuT8u7Fh7i7Pi1lR2pbIO5sL4PqFif/qYBFY0Au0LN57X+rtxV69ReExNO+kximzAp8 +eng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=29ZV4GOZT/03Bjf+MyTwKlvuQUYq4U6PK43stGTt/dg=; b=dOKewC+H3KeGheyCDfhS66+e8QI2BZXRQAUmkSpxTjNpXP7q+pjMcrZnAU+fBVMUQe xyAVD22bzH7vA3dvv08jfstgWl+7ZKM46ihOOGfimcD5Z4PBBJgxoqlxa8f5kn16VpkH r95rurV4olddiZ3XMQKvaW/c5bGj+o6ApOPfL3u6JapWukou0xrklT9q3pPGp0ad2wsS ls0A4dbzNxke1WCkqlLVEvUhWHAjEeCeDxhiSsJF/pj8e1CFFvMdTSFIu2iPSETMhn2o EP3l0DsFc9Vh8qWv7hC0bdduPzmPdc5GeVhI7Em4myr+B5LgNtZjfzEnVGulCC+8JxO6 SxMw== X-Gm-Message-State: ALQs6tD4ePgBHeSK9lqGGQMoNOYGcPxudh+3azTCCQPiJgDpJlFb/ZCo A717Zb64+5ah0AAr87WJegzlXDChzI+ow9QjGQk= X-Google-Smtp-Source: AB8JxZqhwJlAU90jQ+C0yGMryMHA352qT/aXvdmmbbB8R6mDXPadoYTXqpqsjHCguC+MYYvikqr2nA4NxSzZcw5DbHU= X-Received: by 2002:a19:7390:: with SMTP id h16-v6mr12242394lfk.67.1525258225126; Wed, 02 May 2018 03:50:25 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a19:a24a:0:0:0:0:0 with HTTP; Wed, 2 May 2018 03:50:24 -0700 (PDT) In-Reply-To: References: From: Pedro Larroy Date: Wed, 2 May 2018 12:50:24 +0200 Message-ID: Subject: Re: The New Scala API To: dev@mxnet.incubator.apache.org Content-Type: multipart/alternative; boundary="000000000000b311c6056b36dafd" --000000000000b311c6056b36dafd Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi I had a brief look and I have some comments: 1 - Excessive use of Option: Having an optional map is often not necessary. What's the semantic difference between an empty map and passing None? What about a variable argument list of Pairs like: (attr: Pair[String,Sring]*) it can be then called like ("lr" -> "0.01", "activation" -> "relu") with variable number of arguments. null is definitely bad, but we can have an empty map or other default arguments that make sense in some cases, like an empty string for name instead of an optional string. I'm very much pro Option instead of null but it adds clunkiness that is not always necessary, as explained. 2 - One of the fundamental problems is that the arguments are untyped, of type String -> String, I don't see this solved in this proposal. Is it possible to do something better, ie. have typed arguments? when we extend dmlc::Parameter we actually know the C++ type, we could do something smart about this. For example using clang to extract the types and generate the API with proper types. Pedro. On Wed, May 2, 2018 at 4:01 AM, Naveen Swamy wrote: > No, the implementation still comes from MXNet backend. > I don't think there would be any overhead unlike C++ with virtual > functions, Interface or trait is just telling you what the signature look= s > like. users program against interfaces and can choose a different > implementation if they like. For example(only to explain the topic) if we > had different implementations for CPU and GPU, they could simply choose t= he > version of Implementation they like. To visualize think of a bridge desig= n > pattern. > > > > On Tue, May 1, 2018 at 6:49 PM, Marco de Abreu < > marco.g.abreu@googlemail.com > > wrote: > > > Hey Naveen, > > > > I'm not familiar with Scala, so please correct me if I'm wrong with my > > assumptions. This basically sounds like we'd have an interface for the > > Scala API and a separate implementation, right? > > > > In general, I really like these decoupled approaches (you already > > highlighted the advantages)! It would introduce a small overhead since > we'd > > have to define the traits and implementation at two (different) > locations, > > but it definitely increases clarity and separates concerns. I don't thi= nk > > we will have multiple implementations in future, but this might allow u= s > to > > define the user-facing API in form of traits while allowing us to chang= e > > the underlying "glue" as we want. > > > > My only slight concern is that this would introduce a bit more overhead > > since the documentation and API are defined somewhere else, but I have = no > > experience with Scala, so I'll leave this up to the people who are more > > familiar with the topic. > > > > Best regards, > > Marco > > > > On Tue, May 1, 2018 at 11:48 PM, Naveen Swamy > wrote: > > > > > I think this project is going to significantly improve the usability = of > > > MXNet-Scala APIs. > > > > > > I had a offline discussion with Qing about this. I agree with Yizhi > that > > > keeping in the same namespace will make it easy for existing and new > > users > > > to quickly lookup the APIs. Along with that I have one recommendation= , > > > first generate a trait for all the APIs and have the the instance in > the > > > existing symbol. The usage would be something like below: > > > > > > trait SymbolAPIBase { > > > /** > > > api doc > > > */ > > > def Activation() > > > } > > > > > > object SymbolAPI extends SymbolAPIBase { > > > def Activation() > > > } > > > > > > object Symbol { > > > val api: SymbolBase =3D new SymbolAPI() > > > } > > > > > > Now users would use it as > > > Symbol.api.Activation() > > > > > > Creating a trait/Interface as several advantages: > > > 1) User do not have to know which concrete class to use, can be decid= ed > > > later(Dependency Injection or Configuration driven application) > > > 2) Easier to change the implementation later without breaking the use= r > > code > > > -> You can insert different implementations through configuration usi= ng > > > frameworks such as Spring, etc., -> This is standard in most JVM driv= en > > > projects > > > 3) Easier to Unit test -> You can create Mocks easily using Mockito > > > 4) helps with multiple inheritance=E2=80=A6you cannot inherit multipl= e classes > > > > > > Let me know what do you guys think. > > > > > > Thanks, Naveen > > > > > > On Sun, Apr 22, 2018 at 9:09 PM, YiZhi Liu > wrote: > > > > > > > I personally like the design here. Since we have seen technical > > > > difficulties of compatibility, I would like to ask people pay > > > > attention to the 'How to combine with existing APIs' section: > > > > https://cwiki.apache.org/confluence/display/MXNET/ > > > > MXNet+Scala+API+Usability+Improvement#MXNetScalaAPIUsabilityImprove= m > > ent- > > > > HowtocombinewithexistingAPIs > > > > > > > > Qing proposed three options, > > > > > > > > 1. Add a new Class/Object called "NewSymbol/NDArray" with full > > > > implementation. > > > > 2. Create a new Class and change the name space for all of the > > > > functions (e.g Activation -> NewActivation) and let Symbol/NDArray > > > > extends that. > > > > 3. Create a new Class and override the Same functions with differen= t > > > > implementations. > > > > > > > > If we have to choose from option 1 and 2, I would like to +0.5 for > > > > option 2, with which users can quickly aware of the new easy-to-use > > > > API: they type 'Symbol.' in IDE as usual and these functions pop up= . > > > > > > > > 2018-04-19 10:58 GMT-07:00 Qing Lan : > > > > > Hi All, > > > > > > > > > > I am Qing, one of the Scala API maintainer for MXNet. I would lik= e > to > > > > propose a new design on Scala APIs, it will be really helpful for > user > > to > > > > use MXNet Symbol/NDArray. This is a follow-up from Naveen=E2=80=99s= proposal. > > > > > > > > > > Background: > > > > > The current design on Scala would take arguments as key-value pai= r > > and > > > > didn=E2=80=99t provide the type information for different arguments= . There > are > > > > document missing for different functions which makes it even hard t= o > > use. > > > > > > > > > > Our approach: > > > > > We will provide a better designed Scala API for user to use with > full > > > > documentation and arguments definition. All arguments will be > > > specifically > > > > targeted to different functions. Please see one example that we sho= w > in > > > the > > > > Wiki > > > MXNet+Scala+API+Usability+Improvement> and leave any thoughts you > may > > > > have. This wiki includes examples, targets and scenarios we have so > > far. > > > > > > > > > > Thanks, > > > > > Qing > > > > > > > > > > > > > > > > -- > > > > Yizhi Liu > > > > DMLC member > > > > Amazon Web Services > > > > Vancouver, Canada > > > > > > > > > > --000000000000b311c6056b36dafd--