mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunanmcg...@gmail.com>
Subject Re: Java API for MXNet
Date Wed, 16 Aug 2017 16:45:02 GMT
I don't think there will be problems under "11", did the user see concrete
errors?

Best,

Nan



On Wed, Aug 16, 2017 at 9:30 AM, YiZhi Liu <javelinjs@gmail.com> wrote:

> Hi Nan,
>
> Users have 2.11, but with a different minor version, will it cause
> conflicts?
>
> 2017-08-17 0:19 GMT+08:00 Nan Zhu <zhunanmcgill@gmail.com>:
> > Hi, Yizhi,
> >
> > You mean users have 2.10 env while we assemble 2.11 in it?
> >
> > Best,
> >
> > Nan
> >
> > On Wed, Aug 16, 2017 at 9:08 AM, YiZhi Liu <javelinjs@gmail.com> wrote:
> >
> >> Hi Joern,
> >>
> >> The point is that, the front is not a simple wrapper of c_api.h, as
> >> you mentioned, which can be easily achieved by JavaCPP.
> >>
> >> I have noticed the potential conflicts between the assembled scala
> >> library and the one in users' environment. Can we remove the scala
> >> library from the assembly jar? @Nan It wouldn't be a problem since the
> >> scala libraries with same major version are compatible.
> >>
> >> 2017-08-16 23:49 GMT+08:00 Joern Kottmann <kottmann@gmail.com>:
> >> > Hello,
> >> >
> >> > I personally had quite some issues with Scala dependencies in
> >> > different versions and Spark, where one version is not compatible with
> >> > the other version. Then you need to debug the dependency tree to find
> >> > the places where the versions don't match. Every project which would
> >> > like to use MXnet then has to depend on Scala and might also get
> >> > conflicts if other dependencies depend on different Scala versions.
> >> > Probably something which will cause issues for some of your users.
> >> > Users who want to use Java might not be familiar with Scala dependency
> >> > problems and have a hard time resolving them by getting strange error
> >> > messages.
> >> >
> >> > The JNI layer could be generated with JavaCPP, then we would not need
> >> > to write/maintain the C and the  jvm side for that our self.
> >> > A good example of JavaCPP and Scala usage is Apache Mahout [1].
> >> >
> >> > Even if we don't use JavaCPP, the JNI layer should be easy to get into
> >> > a state where both can share it, the current Scala JNI layers LibInfo
> >> > classes could be converted to Java classes and would in most cases
> >> > require only minor changes in the Scala code.
> >> >
> >> > Jörn
> >> >
> >> > [1] https://github.com/apache/mahout/tree/master/viennacl/src/main
> >> >
> >> > On Wed, Aug 16, 2017 at 5:30 PM, Nan Zhu <zhunanmcgill@gmail.com>
> wrote:
> >> >> I agree with Yizhi
> >> >>
> >> >> My major concern is the duplicate implementations, which are usually
> >> one of
> >> >> the major sources of bugs, especially with two languages which are
> >> >> naturally interactive (OK, Calling Scala from Java might need some
> more
> >> >> efforts). It is just like we provide C++ & C APIs of MxNet in two
> >> separated
> >> >> packages.
> >> >>
> >> >> About dependency problem, when you say "As far as I see this has the
> >> great
> >> >> disadvantage that the Java API would force Scala as a dependency onto
> >> the
> >> >> java users.", would you please give a concrete example causing
> critical
> >> >> issues?
> >> >>
> >> >> Best,
> >> >>
> >> >> Nan
> >> >>
> >> >>
> >> >>
> >> >> On Wed, Aug 16, 2017 at 8:19 AM, YiZhi Liu <javelinjs@gmail.com>
> wrote:
> >> >>
> >> >>> Hi,
> >> >>>
> >> >>> If we build the Java API from the very beginning, i.e. the JNI
part,
> >> >>> we have to rewrite the codes for training, predict, inferShape,
etc.
> >> >>> It would be too heavy to maintain a totally new front language.
> >> >>>
> >> >>> As far as I see, I don't think Scala library dependency would be
a
> big
> >> >>> problem in most cases, unless we are going to use it in embedded
> >> >>> devices. Could you illustrate some use-cases where you cannot
> involve
> >> >>> Scala dependencies?
> >> >>>
> >> >>> 2017-08-16 22:13 GMT+08:00 Joern Kottmann <kottmann@gmail.com>:
> >> >>> > Hello,
> >> >>> >
> >> >>> > the approach which is taken by Spark is described here [1].
> >> >>> >
> >> >>> > As far as I see this has the great disadvantage that the Java
API
> >> >>> > would force Scala as a dependency onto the java users.
> >> >>> > For a library it is always a great advantage if it doesn't
have
> many
> >> >>> > dependencies, or zero dependencies. In our case it could be
quite
> >> >>> > realistic to have a thin wrapper around the C API without
needing
> any
> >> >>> > other dependencies (or only dependencies which can't be avoided).
> >> >>> >
> >> >>> > The JNI layer could easily be shared between the Java and
Scala
> API.
> >> >>> > As far as I understand is the JNI layer in the Scala API anyway
> >> >>> > private and a change to it wouldn't require that the public
part
> of
> >> >>> > the Scala API is changed.
> >> >>> >
> >> >>> > What do you think?
> >> >>> >
> >> >>> > Jörn
> >> >>> >
> >> >>> > [1] https://cwiki.apache.org/confluence/display/SPARK/Java+
> >> API+Internals
> >> >>> >
> >> >>> > On Wed, Aug 16, 2017 at 3:39 PM, YiZhi Liu <javelinjs@gmail.com>
> >> wrote:
> >> >>> >> Hi Joern,
> >> >>> >>
> >> >>> >> I suggest to build Java API as a wrapper of Scala API,
re-use
> most
> >> of
> >> >>> >> the procedures. Referring to the Java API in Apache Spark.
> >> >>> >>
> >> >>> >> 2017-08-16 18:21 GMT+08:00 Joern Kottmann <joern@apache.org>:
> >> >>> >>> Hello all,
> >> >>> >>>
> >> >>> >>> I would like to propose the addition of a Java API
to MXNet.
> >> >>> >>>
> >> >>> >>> There has been some previous work done for the Scala
API, and it
> >> makes
> >> >>> >>> sense to at least share the JNI layer between the
two.
> >> >>> >>>
> >> >>> >>> The Java  API probably should be aligned with the
Python API
> (and
> >> >>> >>> others which exist already) with a few changes to
give it a
> native
> >> >>> >>> Java feel.
> >> >>> >>>
> >> >>> >>> As far as I understand there are multiple people interested
to
> >> work on
> >> >>> >>> this and it would be good to maybe come up with a
written
> proposal
> >> on
> >> >>> >>> how things should be.
> >> >>> >>>
> >> >>> >>> My motivation is to get a Java API which can be used
by Apache
> >> OpenNLP
> >> >>> >>> to solve various NLP tasks using Deep Learning based
approaches
> >> and I
> >> >>> >>> am also interested to work on MXNet.
> >> >>> >>>
> >> >>> >>> Jörn
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >> Yizhi Liu
> >> >>> >> DMLC member
> >> >>> >> Technical Manager
> >> >>> >> Qihoo 360 Inc, Shanghai, China
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Yizhi Liu
> >> >>> DMLC member
> >> >>> Technical Manager
> >> >>> Qihoo 360 Inc, Shanghai, China
> >> >>>
> >>
> >>
> >>
> >> --
> >> Yizhi Liu
> >> DMLC member
> >> Technical Manager
> >> Qihoo 360 Inc, Shanghai, China
> >>
>
>
>
> --
> Yizhi Liu
> DMLC member
> Technical Manager
> Qihoo 360 Inc, Shanghai, China
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message