mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunanmcg...@gmail.com>
Subject Re: Java API for MXNet
Date Wed, 16 Aug 2017 16:18:43 GMT
Hi, Jorn

Thanks for providing the info

My opinion is that,

1. MxNet current relies on Scala 2.11 (having some problems due to the
spark package...I am fixing it), which is a stable release for most of
packages...my personal experience hasn't involved a package which only
provides 2.10 version in these days

2. As Tianqi said, we do not have a perfect solution, either handling
potential dependency issues or dealing with duplicate implementations. Why
not start from something simpler, since in most of cases, dependency issue
has lower probability to happen comparing to the bug in duplicate
implementations


Best,

Nan



On Wed, Aug 16, 2017 at 8:49 AM, Joern Kottmann <kottmann@gmail.com> wrote:

> Hello,
>
> I personally had quite some issues with Scala dependencies in
> different versions and Spark, where one version is not compatible with
> the other version. Then you need to debug the dependency tree to find
> the places where the versions don't match. Every project which would
> like to use MXnet then has to depend on Scala and might also get
> conflicts if other dependencies depend on different Scala versions.
> Probably something which will cause issues for some of your users.
> Users who want to use Java might not be familiar with Scala dependency
> problems and have a hard time resolving them by getting strange error
> messages.
>
> The JNI layer could be generated with JavaCPP, then we would not need
> to write/maintain the C and the  jvm side for that our self.
> A good example of JavaCPP and Scala usage is Apache Mahout [1].
>
> Even if we don't use JavaCPP, the JNI layer should be easy to get into
> a state where both can share it, the current Scala JNI layers LibInfo
> classes could be converted to Java classes and would in most cases
> require only minor changes in the Scala code.
>
> Jörn
>
> [1] https://github.com/apache/mahout/tree/master/viennacl/src/main
>
> On Wed, Aug 16, 2017 at 5:30 PM, Nan Zhu <zhunanmcgill@gmail.com> wrote:
> > I agree with Yizhi
> >
> > My major concern is the duplicate implementations, which are usually one
> of
> > the major sources of bugs, especially with two languages which are
> > naturally interactive (OK, Calling Scala from Java might need some more
> > efforts). It is just like we provide C++ & C APIs of MxNet in two
> separated
> > packages.
> >
> > About dependency problem, when you say "As far as I see this has the
> great
> > disadvantage that the Java API would force Scala as a dependency onto the
> > java users.", would you please give a concrete example causing critical
> > issues?
> >
> > Best,
> >
> > Nan
> >
> >
> >
> > On Wed, Aug 16, 2017 at 8:19 AM, YiZhi Liu <javelinjs@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> If we build the Java API from the very beginning, i.e. the JNI part,
> >> we have to rewrite the codes for training, predict, inferShape, etc.
> >> It would be too heavy to maintain a totally new front language.
> >>
> >> As far as I see, I don't think Scala library dependency would be a big
> >> problem in most cases, unless we are going to use it in embedded
> >> devices. Could you illustrate some use-cases where you cannot involve
> >> Scala dependencies?
> >>
> >> 2017-08-16 22:13 GMT+08:00 Joern Kottmann <kottmann@gmail.com>:
> >> > Hello,
> >> >
> >> > the approach which is taken by Spark is described here [1].
> >> >
> >> > As far as I see this has the great disadvantage that the Java API
> >> > would force Scala as a dependency onto the java users.
> >> > For a library it is always a great advantage if it doesn't have many
> >> > dependencies, or zero dependencies. In our case it could be quite
> >> > realistic to have a thin wrapper around the C API without needing any
> >> > other dependencies (or only dependencies which can't be avoided).
> >> >
> >> > The JNI layer could easily be shared between the Java and Scala API.
> >> > As far as I understand is the JNI layer in the Scala API anyway
> >> > private and a change to it wouldn't require that the public part of
> >> > the Scala API is changed.
> >> >
> >> > What do you think?
> >> >
> >> > Jörn
> >> >
> >> > [1] https://cwiki.apache.org/confluence/display/SPARK/Java+
> API+Internals
> >> >
> >> > On Wed, Aug 16, 2017 at 3:39 PM, YiZhi Liu <javelinjs@gmail.com>
> wrote:
> >> >> Hi Joern,
> >> >>
> >> >> I suggest to build Java API as a wrapper of Scala API, re-use most
of
> >> >> the procedures. Referring to the Java API in Apache Spark.
> >> >>
> >> >> 2017-08-16 18:21 GMT+08:00 Joern Kottmann <joern@apache.org>:
> >> >>> Hello all,
> >> >>>
> >> >>> I would like to propose the addition of a Java API to MXNet.
> >> >>>
> >> >>> There has been some previous work done for the Scala API, and it
> makes
> >> >>> sense to at least share the JNI layer between the two.
> >> >>>
> >> >>> The Java  API probably should be aligned with the Python API (and
> >> >>> others which exist already) with a few changes to give it a native
> >> >>> Java feel.
> >> >>>
> >> >>> As far as I understand there are multiple people interested to
work
> on
> >> >>> this and it would be good to maybe come up with a written proposal
> on
> >> >>> how things should be.
> >> >>>
> >> >>> My motivation is to get a Java API which can be used by Apache
> OpenNLP
> >> >>> to solve various NLP tasks using Deep Learning based approaches
and
> I
> >> >>> am also interested to work on MXNet.
> >> >>>
> >> >>> Jörn
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Yizhi Liu
> >> >> DMLC member
> >> >> Technical Manager
> >> >> Qihoo 360 Inc, Shanghai, China
> >>
> >>
> >>
> >> --
> >> Yizhi Liu
> >> DMLC member
> >> Technical Manager
> >> Qihoo 360 Inc, Shanghai, China
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message