mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kellen sunderland <>
Subject Re: New Java Inference API
Date Thu, 10 May 2018 10:22:30 GMT
Hey Andrew, thanks for the write-up.  I think having a Java binding will be
very useful for enterprise users.  Doc looks good but two things I'm
curious about:

How are you planning to handle thread safe inference?   It'll be great if
you can hide the complexity of dealing with dispatch threading from users.

The other thing I think a solid Java API could provide is a limited number
of dependencies.  There's some simple things we can do to make this happen
(create a statically linked, portable so) but there's also some complexity
around minimizing dependencies MXNet.  For example we'll likely want to
release MKL flavoured binaries, we should have a few versions of CUDA
supported.  We could try and have one version that has an absolute minimum
of dependencies (maybe statically linking with openblas).  It might be good
to document exactly the packages you're planning to release, and give some
more details about what the dependencies for the packages would be.

Many thanks for looking into this, I think it'll be a big improvement for
many of our users.


On Thu, May 10, 2018, 12:57 AM Andrew Ayres <>

> Hi all,
> There has been a lot of interest expressed in having a Java API for doing
> inference. The general idea is that after training a model using python,
> users would like to be able to load the model for inference inside their
> existing production eco-system.
> We've begun exploring a few options for the implementation at <
> >
> and would appreciate any insights/feedback.
> Thanks,
> Andrew

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message