mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joern Kottmann <kottm...@gmail.com>
Subject Re: Java API for MXNet
Date Wed, 16 Aug 2017 15:49:37 GMT
Hello,

I personally had quite some issues with Scala dependencies in
different versions and Spark, where one version is not compatible with
the other version. Then you need to debug the dependency tree to find
the places where the versions don't match. Every project which would
like to use MXnet then has to depend on Scala and might also get
conflicts if other dependencies depend on different Scala versions.
Probably something which will cause issues for some of your users.
Users who want to use Java might not be familiar with Scala dependency
problems and have a hard time resolving them by getting strange error
messages.

The JNI layer could be generated with JavaCPP, then we would not need
to write/maintain the C and the  jvm side for that our self.
A good example of JavaCPP and Scala usage is Apache Mahout [1].

Even if we don't use JavaCPP, the JNI layer should be easy to get into
a state where both can share it, the current Scala JNI layers LibInfo
classes could be converted to Java classes and would in most cases
require only minor changes in the Scala code.

Jörn

[1] https://github.com/apache/mahout/tree/master/viennacl/src/main

On Wed, Aug 16, 2017 at 5:30 PM, Nan Zhu <zhunanmcgill@gmail.com> wrote:
> I agree with Yizhi
>
> My major concern is the duplicate implementations, which are usually one of
> the major sources of bugs, especially with two languages which are
> naturally interactive (OK, Calling Scala from Java might need some more
> efforts). It is just like we provide C++ & C APIs of MxNet in two separated
> packages.
>
> About dependency problem, when you say "As far as I see this has the great
> disadvantage that the Java API would force Scala as a dependency onto the
> java users.", would you please give a concrete example causing critical
> issues?
>
> Best,
>
> Nan
>
>
>
> On Wed, Aug 16, 2017 at 8:19 AM, YiZhi Liu <javelinjs@gmail.com> wrote:
>
>> Hi,
>>
>> If we build the Java API from the very beginning, i.e. the JNI part,
>> we have to rewrite the codes for training, predict, inferShape, etc.
>> It would be too heavy to maintain a totally new front language.
>>
>> As far as I see, I don't think Scala library dependency would be a big
>> problem in most cases, unless we are going to use it in embedded
>> devices. Could you illustrate some use-cases where you cannot involve
>> Scala dependencies?
>>
>> 2017-08-16 22:13 GMT+08:00 Joern Kottmann <kottmann@gmail.com>:
>> > Hello,
>> >
>> > the approach which is taken by Spark is described here [1].
>> >
>> > As far as I see this has the great disadvantage that the Java API
>> > would force Scala as a dependency onto the java users.
>> > For a library it is always a great advantage if it doesn't have many
>> > dependencies, or zero dependencies. In our case it could be quite
>> > realistic to have a thin wrapper around the C API without needing any
>> > other dependencies (or only dependencies which can't be avoided).
>> >
>> > The JNI layer could easily be shared between the Java and Scala API.
>> > As far as I understand is the JNI layer in the Scala API anyway
>> > private and a change to it wouldn't require that the public part of
>> > the Scala API is changed.
>> >
>> > What do you think?
>> >
>> > Jörn
>> >
>> > [1] https://cwiki.apache.org/confluence/display/SPARK/Java+API+Internals
>> >
>> > On Wed, Aug 16, 2017 at 3:39 PM, YiZhi Liu <javelinjs@gmail.com> wrote:
>> >> Hi Joern,
>> >>
>> >> I suggest to build Java API as a wrapper of Scala API, re-use most of
>> >> the procedures. Referring to the Java API in Apache Spark.
>> >>
>> >> 2017-08-16 18:21 GMT+08:00 Joern Kottmann <joern@apache.org>:
>> >>> Hello all,
>> >>>
>> >>> I would like to propose the addition of a Java API to MXNet.
>> >>>
>> >>> There has been some previous work done for the Scala API, and it makes
>> >>> sense to at least share the JNI layer between the two.
>> >>>
>> >>> The Java  API probably should be aligned with the Python API (and
>> >>> others which exist already) with a few changes to give it a native
>> >>> Java feel.
>> >>>
>> >>> As far as I understand there are multiple people interested to work
on
>> >>> this and it would be good to maybe come up with a written proposal on
>> >>> how things should be.
>> >>>
>> >>> My motivation is to get a Java API which can be used by Apache OpenNLP
>> >>> to solve various NLP tasks using Deep Learning based approaches and
I
>> >>> am also interested to work on MXNet.
>> >>>
>> >>> Jörn
>> >>
>> >>
>> >>
>> >> --
>> >> Yizhi Liu
>> >> DMLC member
>> >> Technical Manager
>> >> Qihoo 360 Inc, Shanghai, China
>>
>>
>>
>> --
>> Yizhi Liu
>> DMLC member
>> Technical Manager
>> Qihoo 360 Inc, Shanghai, China
>>

Mime
View raw message