mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YiZhi Liu <javeli...@gmail.com>
Subject Re: Java API for MXNet
Date Wed, 16 Aug 2017 16:30:32 GMT
Hi Nan,

Users have 2.11, but with a different minor version, will it cause conflicts?

2017-08-17 0:19 GMT+08:00 Nan Zhu <zhunanmcgill@gmail.com>:
> Hi, Yizhi,
>
> You mean users have 2.10 env while we assemble 2.11 in it?
>
> Best,
>
> Nan
>
> On Wed, Aug 16, 2017 at 9:08 AM, YiZhi Liu <javelinjs@gmail.com> wrote:
>
>> Hi Joern,
>>
>> The point is that, the front is not a simple wrapper of c_api.h, as
>> you mentioned, which can be easily achieved by JavaCPP.
>>
>> I have noticed the potential conflicts between the assembled scala
>> library and the one in users' environment. Can we remove the scala
>> library from the assembly jar? @Nan It wouldn't be a problem since the
>> scala libraries with same major version are compatible.
>>
>> 2017-08-16 23:49 GMT+08:00 Joern Kottmann <kottmann@gmail.com>:
>> > Hello,
>> >
>> > I personally had quite some issues with Scala dependencies in
>> > different versions and Spark, where one version is not compatible with
>> > the other version. Then you need to debug the dependency tree to find
>> > the places where the versions don't match. Every project which would
>> > like to use MXnet then has to depend on Scala and might also get
>> > conflicts if other dependencies depend on different Scala versions.
>> > Probably something which will cause issues for some of your users.
>> > Users who want to use Java might not be familiar with Scala dependency
>> > problems and have a hard time resolving them by getting strange error
>> > messages.
>> >
>> > The JNI layer could be generated with JavaCPP, then we would not need
>> > to write/maintain the C and the  jvm side for that our self.
>> > A good example of JavaCPP and Scala usage is Apache Mahout [1].
>> >
>> > Even if we don't use JavaCPP, the JNI layer should be easy to get into
>> > a state where both can share it, the current Scala JNI layers LibInfo
>> > classes could be converted to Java classes and would in most cases
>> > require only minor changes in the Scala code.
>> >
>> > Jörn
>> >
>> > [1] https://github.com/apache/mahout/tree/master/viennacl/src/main
>> >
>> > On Wed, Aug 16, 2017 at 5:30 PM, Nan Zhu <zhunanmcgill@gmail.com> wrote:
>> >> I agree with Yizhi
>> >>
>> >> My major concern is the duplicate implementations, which are usually
>> one of
>> >> the major sources of bugs, especially with two languages which are
>> >> naturally interactive (OK, Calling Scala from Java might need some more
>> >> efforts). It is just like we provide C++ & C APIs of MxNet in two
>> separated
>> >> packages.
>> >>
>> >> About dependency problem, when you say "As far as I see this has the
>> great
>> >> disadvantage that the Java API would force Scala as a dependency onto
>> the
>> >> java users.", would you please give a concrete example causing critical
>> >> issues?
>> >>
>> >> Best,
>> >>
>> >> Nan
>> >>
>> >>
>> >>
>> >> On Wed, Aug 16, 2017 at 8:19 AM, YiZhi Liu <javelinjs@gmail.com> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> If we build the Java API from the very beginning, i.e. the JNI part,
>> >>> we have to rewrite the codes for training, predict, inferShape, etc.
>> >>> It would be too heavy to maintain a totally new front language.
>> >>>
>> >>> As far as I see, I don't think Scala library dependency would be a big
>> >>> problem in most cases, unless we are going to use it in embedded
>> >>> devices. Could you illustrate some use-cases where you cannot involve
>> >>> Scala dependencies?
>> >>>
>> >>> 2017-08-16 22:13 GMT+08:00 Joern Kottmann <kottmann@gmail.com>:
>> >>> > Hello,
>> >>> >
>> >>> > the approach which is taken by Spark is described here [1].
>> >>> >
>> >>> > As far as I see this has the great disadvantage that the Java API
>> >>> > would force Scala as a dependency onto the java users.
>> >>> > For a library it is always a great advantage if it doesn't have
many
>> >>> > dependencies, or zero dependencies. In our case it could be quite
>> >>> > realistic to have a thin wrapper around the C API without needing
any
>> >>> > other dependencies (or only dependencies which can't be avoided).
>> >>> >
>> >>> > The JNI layer could easily be shared between the Java and Scala
API.
>> >>> > As far as I understand is the JNI layer in the Scala API anyway
>> >>> > private and a change to it wouldn't require that the public part
of
>> >>> > the Scala API is changed.
>> >>> >
>> >>> > What do you think?
>> >>> >
>> >>> > Jörn
>> >>> >
>> >>> > [1] https://cwiki.apache.org/confluence/display/SPARK/Java+
>> API+Internals
>> >>> >
>> >>> > On Wed, Aug 16, 2017 at 3:39 PM, YiZhi Liu <javelinjs@gmail.com>
>> wrote:
>> >>> >> Hi Joern,
>> >>> >>
>> >>> >> I suggest to build Java API as a wrapper of Scala API, re-use
most
>> of
>> >>> >> the procedures. Referring to the Java API in Apache Spark.
>> >>> >>
>> >>> >> 2017-08-16 18:21 GMT+08:00 Joern Kottmann <joern@apache.org>:
>> >>> >>> Hello all,
>> >>> >>>
>> >>> >>> I would like to propose the addition of a Java API to MXNet.
>> >>> >>>
>> >>> >>> There has been some previous work done for the Scala API,
and it
>> makes
>> >>> >>> sense to at least share the JNI layer between the two.
>> >>> >>>
>> >>> >>> The Java  API probably should be aligned with the Python
API (and
>> >>> >>> others which exist already) with a few changes to give
it a native
>> >>> >>> Java feel.
>> >>> >>>
>> >>> >>> As far as I understand there are multiple people interested
to
>> work on
>> >>> >>> this and it would be good to maybe come up with a written
proposal
>> on
>> >>> >>> how things should be.
>> >>> >>>
>> >>> >>> My motivation is to get a Java API which can be used by
Apache
>> OpenNLP
>> >>> >>> to solve various NLP tasks using Deep Learning based approaches
>> and I
>> >>> >>> am also interested to work on MXNet.
>> >>> >>>
>> >>> >>> Jörn
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Yizhi Liu
>> >>> >> DMLC member
>> >>> >> Technical Manager
>> >>> >> Qihoo 360 Inc, Shanghai, China
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Yizhi Liu
>> >>> DMLC member
>> >>> Technical Manager
>> >>> Qihoo 360 Inc, Shanghai, China
>> >>>
>>
>>
>>
>> --
>> Yizhi Liu
>> DMLC member
>> Technical Manager
>> Qihoo 360 Inc, Shanghai, China
>>



-- 
Yizhi Liu
DMLC member
Technical Manager
Qihoo 360 Inc, Shanghai, China

Mime
View raw message