mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunanmcg...@gmail.com>
Subject Re: Java API for MXNet
Date Wed, 16 Aug 2017 16:19:19 GMT
Hi, Yizhi,

You mean users have 2.10 env while we assemble 2.11 in it?

Best,

Nan

On Wed, Aug 16, 2017 at 9:08 AM, YiZhi Liu <javelinjs@gmail.com> wrote:

> Hi Joern,
>
> The point is that, the front is not a simple wrapper of c_api.h, as
> you mentioned, which can be easily achieved by JavaCPP.
>
> I have noticed the potential conflicts between the assembled scala
> library and the one in users' environment. Can we remove the scala
> library from the assembly jar? @Nan It wouldn't be a problem since the
> scala libraries with same major version are compatible.
>
> 2017-08-16 23:49 GMT+08:00 Joern Kottmann <kottmann@gmail.com>:
> > Hello,
> >
> > I personally had quite some issues with Scala dependencies in
> > different versions and Spark, where one version is not compatible with
> > the other version. Then you need to debug the dependency tree to find
> > the places where the versions don't match. Every project which would
> > like to use MXnet then has to depend on Scala and might also get
> > conflicts if other dependencies depend on different Scala versions.
> > Probably something which will cause issues for some of your users.
> > Users who want to use Java might not be familiar with Scala dependency
> > problems and have a hard time resolving them by getting strange error
> > messages.
> >
> > The JNI layer could be generated with JavaCPP, then we would not need
> > to write/maintain the C and the  jvm side for that our self.
> > A good example of JavaCPP and Scala usage is Apache Mahout [1].
> >
> > Even if we don't use JavaCPP, the JNI layer should be easy to get into
> > a state where both can share it, the current Scala JNI layers LibInfo
> > classes could be converted to Java classes and would in most cases
> > require only minor changes in the Scala code.
> >
> > Jörn
> >
> > [1] https://github.com/apache/mahout/tree/master/viennacl/src/main
> >
> > On Wed, Aug 16, 2017 at 5:30 PM, Nan Zhu <zhunanmcgill@gmail.com> wrote:
> >> I agree with Yizhi
> >>
> >> My major concern is the duplicate implementations, which are usually
> one of
> >> the major sources of bugs, especially with two languages which are
> >> naturally interactive (OK, Calling Scala from Java might need some more
> >> efforts). It is just like we provide C++ & C APIs of MxNet in two
> separated
> >> packages.
> >>
> >> About dependency problem, when you say "As far as I see this has the
> great
> >> disadvantage that the Java API would force Scala as a dependency onto
> the
> >> java users.", would you please give a concrete example causing critical
> >> issues?
> >>
> >> Best,
> >>
> >> Nan
> >>
> >>
> >>
> >> On Wed, Aug 16, 2017 at 8:19 AM, YiZhi Liu <javelinjs@gmail.com> wrote:
> >>
> >>> Hi,
> >>>
> >>> If we build the Java API from the very beginning, i.e. the JNI part,
> >>> we have to rewrite the codes for training, predict, inferShape, etc.
> >>> It would be too heavy to maintain a totally new front language.
> >>>
> >>> As far as I see, I don't think Scala library dependency would be a big
> >>> problem in most cases, unless we are going to use it in embedded
> >>> devices. Could you illustrate some use-cases where you cannot involve
> >>> Scala dependencies?
> >>>
> >>> 2017-08-16 22:13 GMT+08:00 Joern Kottmann <kottmann@gmail.com>:
> >>> > Hello,
> >>> >
> >>> > the approach which is taken by Spark is described here [1].
> >>> >
> >>> > As far as I see this has the great disadvantage that the Java API
> >>> > would force Scala as a dependency onto the java users.
> >>> > For a library it is always a great advantage if it doesn't have many
> >>> > dependencies, or zero dependencies. In our case it could be quite
> >>> > realistic to have a thin wrapper around the C API without needing any
> >>> > other dependencies (or only dependencies which can't be avoided).
> >>> >
> >>> > The JNI layer could easily be shared between the Java and Scala API.
> >>> > As far as I understand is the JNI layer in the Scala API anyway
> >>> > private and a change to it wouldn't require that the public part of
> >>> > the Scala API is changed.
> >>> >
> >>> > What do you think?
> >>> >
> >>> > Jörn
> >>> >
> >>> > [1] https://cwiki.apache.org/confluence/display/SPARK/Java+
> API+Internals
> >>> >
> >>> > On Wed, Aug 16, 2017 at 3:39 PM, YiZhi Liu <javelinjs@gmail.com>
> wrote:
> >>> >> Hi Joern,
> >>> >>
> >>> >> I suggest to build Java API as a wrapper of Scala API, re-use most
> of
> >>> >> the procedures. Referring to the Java API in Apache Spark.
> >>> >>
> >>> >> 2017-08-16 18:21 GMT+08:00 Joern Kottmann <joern@apache.org>:
> >>> >>> Hello all,
> >>> >>>
> >>> >>> I would like to propose the addition of a Java API to MXNet.
> >>> >>>
> >>> >>> There has been some previous work done for the Scala API, and
it
> makes
> >>> >>> sense to at least share the JNI layer between the two.
> >>> >>>
> >>> >>> The Java  API probably should be aligned with the Python API
(and
> >>> >>> others which exist already) with a few changes to give it a
native
> >>> >>> Java feel.
> >>> >>>
> >>> >>> As far as I understand there are multiple people interested
to
> work on
> >>> >>> this and it would be good to maybe come up with a written proposal
> on
> >>> >>> how things should be.
> >>> >>>
> >>> >>> My motivation is to get a Java API which can be used by Apache
> OpenNLP
> >>> >>> to solve various NLP tasks using Deep Learning based approaches
> and I
> >>> >>> am also interested to work on MXNet.
> >>> >>>
> >>> >>> Jörn
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Yizhi Liu
> >>> >> DMLC member
> >>> >> Technical Manager
> >>> >> Qihoo 360 Inc, Shanghai, China
> >>>
> >>>
> >>>
> >>> --
> >>> Yizhi Liu
> >>> DMLC member
> >>> Technical Manager
> >>> Qihoo 360 Inc, Shanghai, China
> >>>
>
>
>
> --
> Yizhi Liu
> DMLC member
> Technical Manager
> Qihoo 360 Inc, Shanghai, China
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message