mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samuel Audet <notificati...@github.com>
Subject Re: [apache/incubator-mxnet] [RFC] MXNet 2.0 JVM Language development (#17783)
Date Sun, 26 Jul 2020 00:43:42 GMT
> ## What's missing
> 
> javacpp-presets-mxnet doesn't expose APIs form nnvm/c_api.h (some of current python/gluon
API depends on APIs in nnvm/c_api.h)

I've added that the other day, thanks to @frankfliu for pointing this out: https://github.com/bytedeco/javacpp-presets/commit/976e6f7d307b3f3855f39413c494d8f482c9adf6

> See javadoc: http://bytedeco.org/javacpp-presets/mxnet/apidocs/
> 
> 1. Java class name is “mxnet”, which is not following java naming conventions

That's not hardcoded. We can use whatever name we want for that class.

> 2. Each pointer has a corresponding java class, which is arguable. It's necessary to
expose them as strong type class if they meant to be used directly by end developer. But they
really should only be internal implementation of the API. It's overkill to expose them as
a Type instead of just a pointer.

We can map everything to `Pointer`, that's not a problem either.

> 3. All the classes (except mxnet.java) are hand written.

No, they are not. Everything in the `src/gen` directory here is generated at build time:
https://github.com/bytedeco/javacpp-presets/tree/master/mxnet/src/gen/java/org/bytedeco/mxnet

> 4. API mapping are hand coded as well.

If you're talking about this file, yes, that's the only thing that is written manually:
https://github.com/bytedeco/javacpp-presets/blob/master/mxnet/src/main/java/org/bytedeco/mxnet/presets/mxnet.java
(The formatting is a bit crappy, I haven't touched it in a while, but we can make it look
prettier like this:
https://github.com/bytedeco/javacpp-presets/blob/master/onnxruntime/src/main/java/org/bytedeco/onnxruntime/presets/onnxruntime.java
)

> ## Performance
> 
> JavaCPP native library load takes a long time, it takes average _2.6 seconds_ to initialize
libmxnet.so with javacpp.
> 
> Loader.load(org.bytedeco.mxnet.global.mxnet.class);

Something's wrong, that takes less than 500 ms on my laptop, and that includes loading OpenBLAS,
OpenCV, and a lookup for CUDA and MKL, which can obviously be optimized... In any case, we
can debug that later to see what is going wrong on your end.

> ## Issues
> 
> The open source code on github doesn't match the binary release on maven central:
> 
> * the maven group and the java package name are different.

Both the group ID and the package names are `org.bytedeco`, but in any case, if that gets
maintained somewhere here, I imagine it would be changed to something like `org.apache.mxnet.xyz.internal.etc`

> * c predict API is not included in maven version

Yes it is: http://bytedeco.org/javacpp-presets/mxnet/apidocs/org/bytedeco/mxnet/global/mxnet.html
 
> * Example code doesn't work with maven artifacts, it can only build with snapshot version
locally.

https://github.com/bytedeco/javacpp-presets/tree/master/mxnet/samples works fine for me on
Linux:
```
$ mvn -U clean compile exec:java -Djavacpp.platform.custom -Djavacpp.platform.host -Dexec.args=apple.jpg
...
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet-platform/1.7.0.rc1-1.5.4-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet-platform/1.7.0.rc1-1.5.4-SNAPSHOT/maven-metadata.xml
(1.3 kB at 2.5 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet-platform/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-platform-1.7.0.rc1-1.5.4-20200725.115300-20.pom
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet-platform/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-platform-1.7.0.rc1-1.5.4-20200725.115300-20.pom
(4.7 kB at 9.3 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-presets/1.5.4-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-presets/1.5.4-SNAPSHOT/maven-metadata.xml
(610 B at 1.5 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-presets/1.5.4-SNAPSHOT/javacpp-presets-1.5.4-20200725.155410-6590.pom
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-presets/1.5.4-SNAPSHOT/javacpp-presets-1.5.4-20200725.155410-6590.pom
(84 kB at 91 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv-platform/4.4.0-1.5.4-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv-platform/4.4.0-1.5.4-SNAPSHOT/maven-metadata.xml
(1.2 kB at 2.6 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv-platform/4.4.0-1.5.4-SNAPSHOT/opencv-platform-4.4.0-1.5.4-20200725.082627-40.pom
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv-platform/4.4.0-1.5.4-SNAPSHOT/opencv-platform-4.4.0-1.5.4-20200725.082627-40.pom
(7.9 kB at 19 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas-platform/0.3.10-1.5.4-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas-platform/0.3.10-1.5.4-SNAPSHOT/maven-metadata.xml
(1.3 kB at 2.1 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas-platform/0.3.10-1.5.4-SNAPSHOT/openblas-platform-0.3.10-1.5.4-20200724.193951-177.pom
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas-platform/0.3.10-1.5.4-SNAPSHOT/openblas-platform-0.3.10-1.5.4-20200724.193951-177.pom
(7.9 kB at 16 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-platform/1.5.4-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-platform/1.5.4-SNAPSHOT/maven-metadata.xml
(1.2 kB at 2.8 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-platform/1.5.4-SNAPSHOT/javacpp-platform-1.5.4-20200720.164410-35.pom
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-platform/1.5.4-SNAPSHOT/javacpp-platform-1.5.4-20200720.164410-35.pom
(60 kB at 112 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp/1.5.4-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp/1.5.4-SNAPSHOT/maven-metadata.xml
(4.3 kB at 8.1 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp/1.5.4-SNAPSHOT/javacpp-1.5.4-20200725.222627-485.pom
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp/1.5.4-SNAPSHOT/javacpp-1.5.4-20200725.222627-485.pom
(20 kB at 52 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas/0.3.10-1.5.4-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas/0.3.10-1.5.4-SNAPSHOT/maven-metadata.xml
(4.2 kB at 6.9 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas/0.3.10-1.5.4-SNAPSHOT/openblas-0.3.10-1.5.4-20200725.222937-191.pom
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas/0.3.10-1.5.4-SNAPSHOT/openblas-0.3.10-1.5.4-20200725.222937-191.pom
(4.8 kB at 9.9 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv/4.4.0-1.5.4-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv/4.4.0-1.5.4-SNAPSHOT/maven-metadata.xml
(4.6 kB at 9.7 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv/4.4.0-1.5.4-SNAPSHOT/opencv-4.4.0-1.5.4-20200725.222953-47.pom
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv/4.4.0-1.5.4-SNAPSHOT/opencv-4.4.0-1.5.4-20200725.222953-47.pom
(11 kB at 23 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/1.7.0.rc1-1.5.4-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/1.7.0.rc1-1.5.4-SNAPSHOT/maven-metadata.xml
(2.6 kB at 5.7 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-1.7.0.rc1-1.5.4-20200725.222844-30.pom
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-1.7.0.rc1-1.5.4-20200725.222844-30.pom
(15 kB at 28 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet-platform/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-platform-1.7.0.rc1-1.5.4-20200725.115300-20.jar
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv/4.4.0-1.5.4-SNAPSHOT/opencv-4.4.0-1.5.4-20200725.222953-47-linux-x86_64.jar
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas-platform/0.3.10-1.5.4-SNAPSHOT/openblas-platform-0.3.10-1.5.4-20200724.193951-177.jar
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv/4.4.0-1.5.4-SNAPSHOT/opencv-4.4.0-1.5.4-20200725.222953-47.jar
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv-platform/4.4.0-1.5.4-SNAPSHOT/opencv-platform-4.4.0-1.5.4-20200725.082627-40.jar
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet-platform/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-platform-1.7.0.rc1-1.5.4-20200725.115300-20.jar
(3.4 kB at 8.6 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-platform/1.5.4-SNAPSHOT/javacpp-platform-1.5.4-20200720.164410-35.jar
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp-platform/1.5.4-SNAPSHOT/javacpp-platform-1.5.4-20200720.164410-35.jar
(6.1 kB at 7.7 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp/1.5.4-SNAPSHOT/javacpp-1.5.4-20200725.222627-485-linux-x86_64.jar
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv-platform/4.4.0-1.5.4-SNAPSHOT/opencv-platform-4.4.0-1.5.4-20200725.082627-40.jar
(3.6 kB at 3.4 kB/s)
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas-platform/0.3.10-1.5.4-SNAPSHOT/openblas-platform-0.3.10-1.5.4-20200724.193951-177.jar
(3.6 kB at 3.4 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas/0.3.10-1.5.4-SNAPSHOT/openblas-0.3.10-1.5.4-20200725.222937-191.jar
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas/0.3.10-1.5.4-SNAPSHOT/openblas-0.3.10-1.5.4-20200725.222937-191-linux-x86_64.jar
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp/1.5.4-SNAPSHOT/javacpp-1.5.4-20200725.222627-485-linux-x86_64.jar
(25 kB at 21 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-1.7.0.rc1-1.5.4-20200725.222844-30.jar
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas/0.3.10-1.5.4-SNAPSHOT/openblas-0.3.10-1.5.4-20200725.222937-191.jar
(170 kB at 76 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp/1.5.4-SNAPSHOT/javacpp-1.5.4-20200725.222627-485.jar
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/javacpp/1.5.4-SNAPSHOT/javacpp-1.5.4-20200725.222627-485.jar
(467 kB at 151 kB/s)
Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-1.7.0.rc1-1.5.4-20200725.222844-30-linux-x86_64.jar
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv/4.4.0-1.5.4-SNAPSHOT/opencv-4.4.0-1.5.4-20200725.222953-47.jar
(1.6 MB at 509 kB/s)
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-1.7.0.rc1-1.5.4-20200725.222844-30.jar
(3.3 MB at 706 kB/s)
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/openblas/0.3.10-1.5.4-SNAPSHOT/openblas-0.3.10-1.5.4-20200725.222937-191-linux-x86_64.jar
(14 MB at 1.7 MB/s)
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/1.7.0.rc1-1.5.4-SNAPSHOT/mxnet-1.7.0.rc1-1.5.4-20200725.222844-30-linux-x86_64.jar
(44 MB at 2.0 MB/s)
Downloaded from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/opencv/4.4.0-1.5.4-SNAPSHOT/opencv-4.4.0-1.5.4-20200725.222953-47-linux-x86_64.jar
(26 MB at 1.1 MB/s)
...
Best Result: Granny Smith (id=948, accuracy=0.96502399)
run successfully
```
What is the error that you're getting? I've also tested on Mac just now and still no problems.

-- 
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-mxnet/issues/17783#issuecomment-663921595
Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message