mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianqi Chen <tqc...@cs.washington.edu>
Subject Re: Request for suggestions- Supporting onnx in mxnet
Date Thu, 19 Oct 2017 00:19:06 GMT
>
> - “More hardware backends to mxnet” – MXNet users get the same benefit of
> HW support implementing ONNX import on top of MXNet symbolic, right?
>

The support of nnvm compiler compilation comes directly from going into
nnvm/top. This include supporting interesting operators onnx do not yet
support(e.g. broadcast arithmetics) and real compilation pipeline to code.


> - “NNVM Compiler now received contributions from AWS, UW and many other
> folks in MXNet community.” – agreed it is ramping up, but when you look at
> the data, it is clear that it is very early on for NNVM. Looking at the
> repo, it has overall 223 commits, 0 releases. Compare it to MXNet with 6136
> commits and 32 releases. It seems to be still early on for NNVM, and for a
> more reliable initial implementation building the import on top of MXNet is
> easier, faster and safer. MXNet has lots of users already using the
> Symbolic API which hopefully mean that is a mature API that is not likely
> to have breaking changes or major issues.
>

One major reason that NNVM itself get less commit, is because it learns
already a lot of lessons from pains we had when building MXNet. Note that
the MXNet's symbolic API itself is built on top of NNVM for more than a
year now.

The only difference between mxnet's current symbolic API and nnvm/top 's
API is:
- MXNet's API contains legacy issues due to backward compatibility, we
might consider deprecate some of them.
- nnvm/top operators do not suffer from legacy issues and strictly follows
convention of numpy and Gluon.
- In that sense, actually nnvm/top's symbolic API is cleaner and more
stable, and is the final form we want to migrate into.

Tianqi


> On 10/18/17, 14:13, "Tianqi Chen" <workcrow@gmail.com on behalf of
> tqchen@cs.washington.edu> wrote:
>
>     I am strongly recommending going through the nnvm/top. One major
> reason in
>     here is that the support of nnvm/top layer NOT ONLY mean compatibility
> of
>     model format with onnx. These are the major benefits:
>
>
>     - More hardware backends to mxnet, including opencl, metal, Raspberry
> Pi,
>     web browser. These things are automatically enabled by going through
> this
>     layer. In general, we design nnvm/tvm stack to resolve the challenge of
>     current mxnet's weakness in terms deploying to more hardware backends.
>
>     - More frontend capabilities, nnvm's gluon style IR ingests now from
>     CoreML, ONNX and in future keras. Supporting those will reduce the
> amount
>     of engineering effort needed.
>
>     - Future compatibility. We all agree that the future being migrated to
>     gluon's API. NNVM/top tries to look ahead by directly adopting the
> symbolic
>     API to be gluon.
>
>
>     I would also like to correct some of the mentioned facts with regard to
>     nnvm/tvm stack
>
>     1.   Nascent project with few contributors
>
>     NNVM Compiler now received contributions from AWS, UW and many other
> folks
>     in MXNet community. NNVM itself is already being used by MXNet.
>     MXNet's internal IR is migrating toward gluon, and its final form being
>     nnvm/top
>
>     3.   Does not support all operators that exist in MXNet Symbolic API
>
>     Neither NNVM/top or onnx support all operators that exist in mxnet
> symbolic
>     API. The end goal here is mainly to make nnvm/top onnx compatible,
> which is
>     a more reasonable goal.
>
>     4.  No CI Pipeline and testcases
>
>     NNVM already contains a compiler contains unittests and ci tested with
>     integration  https://github.com/dmlc/nnvm, with a CI pipline that is
> well
>     tested on CPU and GPU cases for front-ends.
>
>     Tianqi
>
>
>     On Wed, Oct 18, 2017 at 1:41 PM, Roshani Nagmote <
> roshaninagmote2@gmail.com>
>     wrote:
>
>     > Hi guys,
>     >
>     >
>     > I am working on supporting ONNX <https://github.com/onnx/onnx>
> pre-trained
>     > models in Apache MXNet and would like to seek your opinion on the
> choice of
>     > implementation. I also have created a GitHub issue
>     > <https://github.com/apache/incubator-mxnet/issues/8319>. Supporting
> ONNX
>     > in
>     > MXNet will enable users to move between frameworks with their
> models, this
>     > will also enable MXNet project to be a part of the ONNX open
> standard and
>     > steer the direction of ONNX.
>     >
>     >
>     > For those who don’t know ONNX, ONNX is an open source format for AI
> models
>     > which enables models to be transferred between frameworks. Refer to
>     > https://github.com/onnx/onnx for more details.
>     >
>     >
>     > To implement the import/export functionality in MXNet, I propose to
> expose
>     > a MXNet python module “serde”(name taken from Apache Hive project)
> with the
>     > following methods supporting different formats:
>     >
>     > sym, params = mxnet.serde.import(other_format_file,
> other_format=‘onnx’)
>     >
>     > other_format_file =  mxnet.serde.export(mxnet_sym, mxnet_params,
> ‘onnx’)
>     >
>     >
>     > The implementation under the hood can be done in two ways:
>     >
>     >
>     > 1) Implement at the MXNet layer by parsing the ONNX model(in protobuf
>     > format) and turn into MXNet Symbolic operators and build MXNet model
>     > directly. Similarly, I can convert the MXNet model to ONNX format at
> this
>     > layer.
>     >
>     >
>     > 2) The DMLC community has released the nnvm/tvm complier and an
>     > intermediate representation of the models, refer:
>     > http://www.tvmlang.org/2017/10/06/nnvm/tvm-compiler-announce
> ment.html
>     > <http://www.tvmlang.org/2017/10/06/nnvm-compiler-announcement.html>
>     >
>     > Based on the conversation on the GitHub issue
>     > <https://github.com/apache/incubator-mxnet/issues/8319> I opened, Mu
>     > mentioned that MXNet would use nnvm/tvm as the backend in the future.
>     >
>     >
>     > We could hook into this layer to implement the import/export
> functionality.
>     > nnvm/tvm has ONNX 0.1 version import implemented.
>     >
>     > For import,
>     >
>     >    1.
>     >
>     >    I will need to enhance nnvm/tvm’s importer to support ONNX 0.2
>     >    2.
>     >
>     >    Implement nnvm/tvm->mxnet symbolic operators.
>     >
>     > For export:
>     >
>     >
>     >    1.
>     >
>     >    mxnet->nnvm/tvm ( nnvm/tvm provides this implementation already)
>     >    2.
>     >
>     >    I will need to Implement nnvm/tvm>onnx.
>     >
>     >
>     > These are the pros and cons I see in the above approaches:
>     >
>     >    1.
>     >
>     >    Import/export at mxnet layer
>     >
>     > Pros:
>     >
>     >    1.
>     >
>     >    Stable APIs currently used by users.
>     >    2.
>     >
>     >    Larger Apache MXNet community of contributors.
>     >    3.
>     >
>     >    CI pipeline to catch bugs.
>     >    4.
>     >
>     >    Comparatively less time to implement and put it in the hands of
> the
>     >    users.
>     >
>     > Cons:
>     >
>     >    1.
>     >
>     >    In the future we may have to reimplement at the nnvm/tvm layer,
> in case
>     >    MXNet moves to the nnvm/tvm backend(assuming it will move).
>     >
>     >
>     >
>     >    1.
>     >
>     >    Import/export at nnvm/tvm layer
>     >
>     > Pros:
>     >
>     >    1.
>     >
>     >    Less engineering work in case mxnet moves to nnvm/tvm
>     >    2.
>     >
>     >    nnvm/tvm would become a hub to convert to different formats.
>     >    3.
>     >
>     >    nnvm operators are more in parity with mxnet’s gluon APIs this
> could be
>     >    useful in case Gluon becomes the only standard that MXNet will
> support.
>     >
>     > Cons:
>     >
>     >    1.
>     >
>     >    Nascent project with few contributors
>     >    2.
>     >
>     >    Does not support all operators that exist in MXNet Symbolic API
>     >    3.
>     >
>     >    No CI Pipeline
>     >    4.
>     >
>     >    Current Apache MXNet project does not use nnvm/tvm backend
>     >    5.
>     >
>     >    mxnet->nnvm/tvm backend needs more testing and user feedback.
>     >
>     >
>     > Any suggestions on both of these approaches? From user's
> perspective, this
>     > will be an implementation detail that is not exposed.
>     >
>     > Thanks,
>     >
>     > Roshani
>     >
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message