mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carin Meier <carinme...@gmail.com>
Subject Re: Feedback request for new Java API
Date Fri, 28 Sep 2018 14:25:13 GMT
Sorry bad paste on the gist - here is the good one
https://gist.github.com/gigasquid/01cd48f563db4739910592dd9ac9db20

On Fri, Sep 28, 2018 at 10:24 AM Carin Meier <carinmeier@gmail.com> wrote:

> +1 on option #2
>
> In the case of minimizing the the overhead for code maintenance, I wanted
> to suggest the option of investigating generating code from the Java
> Reflection for the Java APIs.  I did a quick gist from Clojure of what the
> generated classes look like from the current Scala Symbol.api for
> FullyConnected here
> https://gist.github.com/gigasquid/01cd48f563db4739910592
>
> I looks like that there is always a base Java class generated will all the
> arguments. If this is the case, then there is a possibility to generate a
> Java api based on this Java method automatically with just a conversion for
> the Scala option and it might be reusable for all the packages.
>
> Not sure if it will work for this use case, but thought I would bring it
> up in case it's helpful.
>
> - Carin
>
> On Fri, Sep 28, 2018 at 7:05 AM Davydenko, Denis <dden@amazon.com.invalid>
> wrote:
>
>> +1 on option #2. Having clear Java interface for NDArray, from my
>> perspective, would be a better experience for Java users as it won't
>> require them to deal with Scala code in any capacity. Overhead of extra
>> code for additional macros is justified, in my mind, as it will be
>> introduced with option #1 either way, just in a different place.
>>
>> --
>> Thanks,
>> Denis
>>
>> On 9/27/18, 6:14 PM, "YiZhi Liu" <eazhi.liu@gmail.com> wrote:
>>
>>     I vote for "2.) Leave the existing macro in place and add another
>>     which generates a Java friendly version"
>>
>>     @Qing @Andrew, could you give some examples, so that people can better
>>     understand how it provides "best possible experience" to Java users.
>>
>>     I have no strong preference between having JavaShape & JavaContext or
>> not.
>>     On Thu, Sep 27, 2018 at 5:56 PM Andrew Ayres <
>> andrew.f.ayres@gmail.com> wrote:
>>     >
>>     > That's not really the conversation I'm wanting to have. I want a
>> discussion
>>     > about the macros with respect to NDArray so that we can get
>> agreement on
>>     > our path forward with respect to implementing the NDArray wrapper.
>>     >
>>     > The design that was put forth and agreed to was for a a Java
>> wrapper around
>>     > the Scala API. Adding a bunch of Java friendly methods inside the
>> Scala
>>     > code would create a mess for users. Maintenance would be
>> essentially the
>>     > same for both because either way you're going to be updating Java
>> methods
>>     > when you make Scala changes.
>>     >
>>     > Let's please stick with the issue in the original email.
>>     >
>>     > Thanks,
>>     > Andrew
>>     >
>>     > On Thu, Sep 27, 2018 at 5:22 PM Qing Lan <lanking520@live.com>
>> wrote:
>>     >
>>     > > I would like to loop this back a layer. Current, there is a
>> discussion in
>>     > > the MXNet Scala community on the ways to implement the Java APIs.
>> Currently
>>     > > there are two thoughts:
>>     > >
>>     > > 1. Make Scala Java Friendly (Create Java compatible methods in
>> the Scala
>>     > > Class. such as NDArray with Java compatible constructor)
>>     > > 2. Make Java friendly wrappers in Scala (Andrew's explanation
>> below)
>>     > >
>>     > > The first approach require minimum input from our side to
>> implement
>>     > > however bring user a bunch of useless api they may not want to
>> use. It also
>>     > > makes Scala package heavier. The good thing is these two packages
>> require
>>     > > minimum maintenance cost. As a tradeoff, if any time in the
>> future we want
>>     > > to make Java big (make Java as the primary language supported by
>> MXNet),
>>     > > then the migration from Scala to Java will be harmful. Spark
>> consider this
>>     > > carefully and decide not to change much on their Scala code base
>> to make it
>>     > > more Java.
>>     > >
>>     > > The second approach will make unique NDArray, Shape, Context and
>> more. The
>>     > > good thing about this is we can always holds a version control on
>> Java.
>>     > > Some breaking changes on Scala may not influence much on Java. It
>> did the
>>     > > best way to decouple the module and good for us to build unique
>> pipeline
>>     > > for Java. The bad thing with this design is the maintenance cost
>> as we need
>>     > > to keep two code bases, but it also make Java side easy to change
>> to make
>>     > > it better compatible with users.
>>     > >
>>     > > Thanks,
>>     > > Qing
>>     > >
>>     > > On 9/27/18, 3:25 PM, "Andrew Ayres" <andrew.f.ayres@gmail.com>
>> wrote:
>>     > >
>>     > >     Hi,
>>     > >
>>     > >     Currently, we're working to implement a new Java API and
>> would like
>>     > > some
>>     > >     feedback from the community on an implementation detail. In
>> short, the
>>     > > new
>>     > >     Java API will use the existing Scala API (in a manner similar
>> to how
>>     > > the
>>     > >     current Clojure API works). This basically means that we're
>> making Java
>>     > >     friendly wrappers to call the existing Scala API.
>>     > >
>>     > >     The feedback we're looking for is on the implementation of
>> NDArray.
>>     > > Scala's
>>     > >     NDArray has a significant amount of code which is generated
>> via macros
>>     > > and
>>     > >     we've got two viable paths to move forward:
>>     > >
>>     > >     1.) Change the macro to generate Java friendly methods  - To
>> do this
>>     > > we'll
>>     > >     modify the macro so that the generated methods won't have
>>     > > default/optional
>>     > >     arguments. There may also have to be some changes to
>> parameter types to
>>     > >     make them Java friendly. The big advantage here is that
>> ongoing
>>     > > maintenance
>>     > >     will easier. The disadvantages are that we'll be changing the
>> existing
>>     > >     Scala NDArray Infer API (it's marked experimental) and Scala
>> users will
>>     > >     lose the ability to use the default and optional arguments.
>>     > >
>>     > >     2.) Leave the existing macro in place and add another which
>> generates a
>>     > >     Java friendly version - The biggest issue here is that we'll
>> be
>>     > > doubling
>>     > >     the number of macros that we've got to maintain. It'll become
>> even more
>>     > >     overhead once we start expanding the Java API with more
>> classes that
>>     > > use
>>     > >     generated code like this. The advantages are that the
>> existing Scala
>>     > >     NDArray Infer API would remain unchanged for Scala users and
>> that the
>>     > > new
>>     > >     macro could be optimized to give the best possible experience
>> to the
>>     > > Java
>>     > >     API.
>>     > >
>>     > >     Thanks,
>>     > >     Andrew
>>     > >
>>     > >
>>     > >
>>
>>
>>
>>     --
>>     Yizhi Liu
>>     DMLC member
>>     Amazon Web Services
>>     Vancouver, Canada
>>
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message