mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Olivier <cjolivie...@gmail.com>
Subject Re: Request for suggestions- Supporting onnx in mxnet
Date Thu, 19 Oct 2017 01:09:32 GMT
My $0.02:

NNVM is not currently an Apache module.  It’s under dmlc umbrella, whose
direction and governance is unclear. For this reason, I am inclined to
support new effort being places in Apache MXNet


-Chris

On Wed, Oct 18, 2017 at 5:19 PM Tianqi Chen <tqchen@cs.washington.edu>
wrote:

> >
> > - “More hardware backends to mxnet” – MXNet users get the same benefit of
> > HW support implementing ONNX import on top of MXNet symbolic, right?
> >
>
> The support of nnvm compiler compilation comes directly from going into
> nnvm/top. This include supporting interesting operators onnx do not yet
> support(e.g. broadcast arithmetics) and real compilation pipeline to code.
>
>
> > - “NNVM Compiler now received contributions from AWS, UW and many other
> > folks in MXNet community.” – agreed it is ramping up, but when you look
> at
> > the data, it is clear that it is very early on for NNVM. Looking at the
> > repo, it has overall 223 commits, 0 releases. Compare it to MXNet with
> 6136
> > commits and 32 releases. It seems to be still early on for NNVM, and for
> a
> > more reliable initial implementation building the import on top of MXNet
> is
> > easier, faster and safer. MXNet has lots of users already using the
> > Symbolic API which hopefully mean that is a mature API that is not likely
> > to have breaking changes or major issues.
> >
>
> One major reason that NNVM itself get less commit, is because it learns
> already a lot of lessons from pains we had when building MXNet. Note that
> the MXNet's symbolic API itself is built on top of NNVM for more than a
> year now.
>
> The only difference between mxnet's current symbolic API and nnvm/top 's
> API is:
> - MXNet's API contains legacy issues due to backward compatibility, we
> might consider deprecate some of them.
> - nnvm/top operators do not suffer from legacy issues and strictly follows
> convention of numpy and Gluon.
> - In that sense, actually nnvm/top's symbolic API is cleaner and more
> stable, and is the final form we want to migrate into.
>
> Tianqi
>
>
> > On 10/18/17, 14:13, "Tianqi Chen" <workcrow@gmail.com on behalf of
> > tqchen@cs.washington.edu> wrote:
> >
> >     I am strongly recommending going through the nnvm/top. One major
> > reason in
> >     here is that the support of nnvm/top layer NOT ONLY mean
> compatibility
> > of
> >     model format with onnx. These are the major benefits:
> >
> >
> >     - More hardware backends to mxnet, including opencl, metal, Raspberry
> > Pi,
> >     web browser. These things are automatically enabled by going through
> > this
> >     layer. In general, we design nnvm/tvm stack to resolve the challenge
> of
> >     current mxnet's weakness in terms deploying to more hardware
> backends.
> >
> >     - More frontend capabilities, nnvm's gluon style IR ingests now from
> >     CoreML, ONNX and in future keras. Supporting those will reduce the
> > amount
> >     of engineering effort needed.
> >
> >     - Future compatibility. We all agree that the future being migrated
> to
> >     gluon's API. NNVM/top tries to look ahead by directly adopting the
> > symbolic
> >     API to be gluon.
> >
> >
> >     I would also like to correct some of the mentioned facts with regard
> to
> >     nnvm/tvm stack
> >
> >     1.   Nascent project with few contributors
> >
> >     NNVM Compiler now received contributions from AWS, UW and many other
> > folks
> >     in MXNet community. NNVM itself is already being used by MXNet.
> >     MXNet's internal IR is migrating toward gluon, and its final form
> being
> >     nnvm/top
> >
> >     3.   Does not support all operators that exist in MXNet Symbolic API
> >
> >     Neither NNVM/top or onnx support all operators that exist in mxnet
> > symbolic
> >     API. The end goal here is mainly to make nnvm/top onnx compatible,
> > which is
> >     a more reasonable goal.
> >
> >     4.  No CI Pipeline and testcases
> >
> >     NNVM already contains a compiler contains unittests and ci tested
> with
> >     integration  https://github.com/dmlc/nnvm, with a CI pipline that is
> > well
> >     tested on CPU and GPU cases for front-ends.
> >
> >     Tianqi
> >
> >
> >     On Wed, Oct 18, 2017 at 1:41 PM, Roshani Nagmote <
> > roshaninagmote2@gmail.com>
> >     wrote:
> >
> >     > Hi guys,
> >     >
> >     >
> >     > I am working on supporting ONNX <https://github.com/onnx/onnx>
> > pre-trained
> >     > models in Apache MXNet and would like to seek your opinion on the
> > choice of
> >     > implementation. I also have created a GitHub issue
> >     > <https://github.com/apache/incubator-mxnet/issues/8319>.
> Supporting
> > ONNX
> >     > in
> >     > MXNet will enable users to move between frameworks with their
> > models, this
> >     > will also enable MXNet project to be a part of the ONNX open
> > standard and
> >     > steer the direction of ONNX.
> >     >
> >     >
> >     > For those who don’t know ONNX, ONNX is an open source format for AI
> > models
> >     > which enables models to be transferred between frameworks. Refer to
> >     > https://github.com/onnx/onnx for more details.
> >     >
> >     >
> >     > To implement the import/export functionality in MXNet, I propose to
> > expose
> >     > a MXNet python module “serde”(name taken from Apache Hive project)
> > with the
> >     > following methods supporting different formats:
> >     >
> >     > sym, params = mxnet.serde.import(other_format_file,
> > other_format=‘onnx’)
> >     >
> >     > other_format_file =  mxnet.serde.export(mxnet_sym, mxnet_params,
> > ‘onnx’)
> >     >
> >     >
> >     > The implementation under the hood can be done in two ways:
> >     >
> >     >
> >     > 1) Implement at the MXNet layer by parsing the ONNX model(in
> protobuf
> >     > format) and turn into MXNet Symbolic operators and build MXNet
> model
> >     > directly. Similarly, I can convert the MXNet model to ONNX format
> at
> > this
> >     > layer.
> >     >
> >     >
> >     > 2) The DMLC community has released the nnvm/tvm complier and an
> >     > intermediate representation of the models, refer:
> >     > http://www.tvmlang.org/2017/10/06/nnvm/tvm-compiler-announce
> > ment.html
> >     > <http://www.tvmlang.org/2017/10/06/nnvm-compiler-announcement.html
> >
> >     >
> >     > Based on the conversation on the GitHub issue
> >     > <https://github.com/apache/incubator-mxnet/issues/8319> I opened,
> Mu
> >     > mentioned that MXNet would use nnvm/tvm as the backend in the
> future.
> >     >
> >     >
> >     > We could hook into this layer to implement the import/export
> > functionality.
> >     > nnvm/tvm has ONNX 0.1 version import implemented.
> >     >
> >     > For import,
> >     >
> >     >    1.
> >     >
> >     >    I will need to enhance nnvm/tvm’s importer to support ONNX 0.2
> >     >    2.
> >     >
> >     >    Implement nnvm/tvm->mxnet symbolic operators.
> >     >
> >     > For export:
> >     >
> >     >
> >     >    1.
> >     >
> >     >    mxnet->nnvm/tvm ( nnvm/tvm provides this implementation already)
> >     >    2.
> >     >
> >     >    I will need to Implement nnvm/tvm>onnx.
> >     >
> >     >
> >     > These are the pros and cons I see in the above approaches:
> >     >
> >     >    1.
> >     >
> >     >    Import/export at mxnet layer
> >     >
> >     > Pros:
> >     >
> >     >    1.
> >     >
> >     >    Stable APIs currently used by users.
> >     >    2.
> >     >
> >     >    Larger Apache MXNet community of contributors.
> >     >    3.
> >     >
> >     >    CI pipeline to catch bugs.
> >     >    4.
> >     >
> >     >    Comparatively less time to implement and put it in the hands of
> > the
> >     >    users.
> >     >
> >     > Cons:
> >     >
> >     >    1.
> >     >
> >     >    In the future we may have to reimplement at the nnvm/tvm layer,
> > in case
> >     >    MXNet moves to the nnvm/tvm backend(assuming it will move).
> >     >
> >     >
> >     >
> >     >    1.
> >     >
> >     >    Import/export at nnvm/tvm layer
> >     >
> >     > Pros:
> >     >
> >     >    1.
> >     >
> >     >    Less engineering work in case mxnet moves to nnvm/tvm
> >     >    2.
> >     >
> >     >    nnvm/tvm would become a hub to convert to different formats.
> >     >    3.
> >     >
> >     >    nnvm operators are more in parity with mxnet’s gluon APIs this
> > could be
> >     >    useful in case Gluon becomes the only standard that MXNet will
> > support.
> >     >
> >     > Cons:
> >     >
> >     >    1.
> >     >
> >     >    Nascent project with few contributors
> >     >    2.
> >     >
> >     >    Does not support all operators that exist in MXNet Symbolic API
> >     >    3.
> >     >
> >     >    No CI Pipeline
> >     >    4.
> >     >
> >     >    Current Apache MXNet project does not use nnvm/tvm backend
> >     >    5.
> >     >
> >     >    mxnet->nnvm/tvm backend needs more testing and user feedback.
> >     >
> >     >
> >     > Any suggestions on both of these approaches? From user's
> > perspective, this
> >     > will be an implementation detail that is not exposed.
> >     >
> >     > Thanks,
> >     >
> >     > Roshani
> >     >
> >
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message