mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carin Meier <carinme...@gmail.com>
Subject Re: Feedback request for new Java API
Date Fri, 28 Sep 2018 14:24:08 GMT
+1 on option #2

In the case of minimizing the the overhead for code maintenance, I wanted
to suggest the option of investigating generating code from the Java
Reflection for the Java APIs.  I did a quick gist from Clojure of what the
generated classes look like from the current Scala Symbol.api for
FullyConnected here https://gist.github.com/gigasquid/01cd48f563db4739910592

I looks like that there is always a base Java class generated will all the
arguments. If this is the case, then there is a possibility to generate a
Java api based on this Java method automatically with just a conversion for
the Scala option and it might be reusable for all the packages.

Not sure if it will work for this use case, but thought I would bring it up
in case it's helpful.

- Carin

On Fri, Sep 28, 2018 at 7:05 AM Davydenko, Denis <dden@amazon.com.invalid>
wrote:

> +1 on option #2. Having clear Java interface for NDArray, from my
> perspective, would be a better experience for Java users as it won't
> require them to deal with Scala code in any capacity. Overhead of extra
> code for additional macros is justified, in my mind, as it will be
> introduced with option #1 either way, just in a different place.
>
> --
> Thanks,
> Denis
>
> On 9/27/18, 6:14 PM, "YiZhi Liu" <eazhi.liu@gmail.com> wrote:
>
>     I vote for "2.) Leave the existing macro in place and add another
>     which generates a Java friendly version"
>
>     @Qing @Andrew, could you give some examples, so that people can better
>     understand how it provides "best possible experience" to Java users.
>
>     I have no strong preference between having JavaShape & JavaContext or
> not.
>     On Thu, Sep 27, 2018 at 5:56 PM Andrew Ayres <andrew.f.ayres@gmail.com>
> wrote:
>     >
>     > That's not really the conversation I'm wanting to have. I want a
> discussion
>     > about the macros with respect to NDArray so that we can get
> agreement on
>     > our path forward with respect to implementing the NDArray wrapper.
>     >
>     > The design that was put forth and agreed to was for a a Java wrapper
> around
>     > the Scala API. Adding a bunch of Java friendly methods inside the
> Scala
>     > code would create a mess for users. Maintenance would be essentially
> the
>     > same for both because either way you're going to be updating Java
> methods
>     > when you make Scala changes.
>     >
>     > Let's please stick with the issue in the original email.
>     >
>     > Thanks,
>     > Andrew
>     >
>     > On Thu, Sep 27, 2018 at 5:22 PM Qing Lan <lanking520@live.com>
> wrote:
>     >
>     > > I would like to loop this back a layer. Current, there is a
> discussion in
>     > > the MXNet Scala community on the ways to implement the Java APIs.
> Currently
>     > > there are two thoughts:
>     > >
>     > > 1. Make Scala Java Friendly (Create Java compatible methods in the
> Scala
>     > > Class. such as NDArray with Java compatible constructor)
>     > > 2. Make Java friendly wrappers in Scala (Andrew's explanation
> below)
>     > >
>     > > The first approach require minimum input from our side to implement
>     > > however bring user a bunch of useless api they may not want to
> use. It also
>     > > makes Scala package heavier. The good thing is these two packages
> require
>     > > minimum maintenance cost. As a tradeoff, if any time in the future
> we want
>     > > to make Java big (make Java as the primary language supported by
> MXNet),
>     > > then the migration from Scala to Java will be harmful. Spark
> consider this
>     > > carefully and decide not to change much on their Scala code base
> to make it
>     > > more Java.
>     > >
>     > > The second approach will make unique NDArray, Shape, Context and
> more. The
>     > > good thing about this is we can always holds a version control on
> Java.
>     > > Some breaking changes on Scala may not influence much on Java. It
> did the
>     > > best way to decouple the module and good for us to build unique
> pipeline
>     > > for Java. The bad thing with this design is the maintenance cost
> as we need
>     > > to keep two code bases, but it also make Java side easy to change
> to make
>     > > it better compatible with users.
>     > >
>     > > Thanks,
>     > > Qing
>     > >
>     > > On 9/27/18, 3:25 PM, "Andrew Ayres" <andrew.f.ayres@gmail.com>
> wrote:
>     > >
>     > >     Hi,
>     > >
>     > >     Currently, we're working to implement a new Java API and would
> like
>     > > some
>     > >     feedback from the community on an implementation detail. In
> short, the
>     > > new
>     > >     Java API will use the existing Scala API (in a manner similar
> to how
>     > > the
>     > >     current Clojure API works). This basically means that we're
> making Java
>     > >     friendly wrappers to call the existing Scala API.
>     > >
>     > >     The feedback we're looking for is on the implementation of
> NDArray.
>     > > Scala's
>     > >     NDArray has a significant amount of code which is generated
> via macros
>     > > and
>     > >     we've got two viable paths to move forward:
>     > >
>     > >     1.) Change the macro to generate Java friendly methods  - To
> do this
>     > > we'll
>     > >     modify the macro so that the generated methods won't have
>     > > default/optional
>     > >     arguments. There may also have to be some changes to parameter
> types to
>     > >     make them Java friendly. The big advantage here is that ongoing
>     > > maintenance
>     > >     will easier. The disadvantages are that we'll be changing the
> existing
>     > >     Scala NDArray Infer API (it's marked experimental) and Scala
> users will
>     > >     lose the ability to use the default and optional arguments.
>     > >
>     > >     2.) Leave the existing macro in place and add another which
> generates a
>     > >     Java friendly version - The biggest issue here is that we'll be
>     > > doubling
>     > >     the number of macros that we've got to maintain. It'll become
> even more
>     > >     overhead once we start expanding the Java API with more
> classes that
>     > > use
>     > >     generated code like this. The advantages are that the existing
> Scala
>     > >     NDArray Infer API would remain unchanged for Scala users and
> that the
>     > > new
>     > >     macro could be optimized to give the best possible experience
> to the
>     > > Java
>     > >     API.
>     > >
>     > >     Thanks,
>     > >     Andrew
>     > >
>     > >
>     > >
>
>
>
>     --
>     Yizhi Liu
>     DMLC member
>     Amazon Web Services
>     Vancouver, Canada
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message