Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mxnet.incubator.apache.org
MIME-Version: 1.0
Sender: workcrow@gmail.com
In-Reply-To: <CAC-xMDL+9_A=LdN1BVNs7Mg28zs_hG8ULsDWBW3kTGw1KQmLKw@mail.gmail.com>
References: <CAC-xMDL+9_A=LdN1BVNs7Mg28zs_hG8ULsDWBW3kTGw1KQmLKw@mail.gmail.com>
From: Tianqi Chen <tqchen@cs.washington.edu>
Date: Wed, 18 Oct 2017 14:13:35 -0700
Message-ID: <CACZfuxZqAHYjX90mKcYBkDXfLPdvKHJLfiwBK38xJnxme-yyDw@mail.gmail.com>
Subject: Re: Request for suggestions- Supporting onnx in mxnet
To: dev@mxnet.incubator.apache.org
Content-Type: multipart/alternative; boundary="94eb2c05d86677e62a055bd8b6a3"
archived-at: Wed, 18 Oct 2017 21:13:41 -0000

--94eb2c05d86677e62a055bd8b6a3
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I am strongly recommending going through the nnvm/top. One major reason in
here is that the support of nnvm/top layer NOT ONLY mean compatibility of
model format with onnx. These are the major benefits:


- More hardware backends to mxnet, including opencl, metal, Raspberry Pi,
web browser. These things are automatically enabled by going through this
layer. In general, we design nnvm/tvm stack to resolve the challenge of
current mxnet's weakness in terms deploying to more hardware backends.

- More frontend capabilities, nnvm's gluon style IR ingests now from
CoreML, ONNX and in future keras. Supporting those will reduce the amount
of engineering effort needed.

- Future compatibility. We all agree that the future being migrated to
gluon's API. NNVM/top tries to look ahead by directly adopting the symbolic
API to be gluon.


I would also like to correct some of the mentioned facts with regard to
nnvm/tvm stack

1.   Nascent project with few contributors

NNVM Compiler now received contributions from AWS, UW and many other folks
in MXNet community. NNVM itself is already being used by MXNet.
MXNet's internal IR is migrating toward gluon, and its final form being
nnvm/top

3.   Does not support all operators that exist in MXNet Symbolic API

Neither NNVM/top or onnx support all operators that exist in mxnet symbolic
API. The end goal here is mainly to make nnvm/top onnx compatible, which is
a more reasonable goal.

4.  No CI Pipeline and testcases

NNVM already contains a compiler contains unittests and ci tested with
integration  https://github.com/dmlc/nnvm, with a CI pipline that is well
tested on CPU and GPU cases for front-ends.

Tianqi


On Wed, Oct 18, 2017 at 1:41 PM, Roshani Nagmote <roshaninagmote2@gmail.com=
>
wrote:

> Hi guys,
>
>
> I am working on supporting ONNX <https://github.com/onnx/onnx> pre-traine=
d
> models in Apache MXNet and would like to seek your opinion on the choice =
of
> implementation. I also have created a GitHub issue
> <https://github.com/apache/incubator-mxnet/issues/8319>. Supporting ONNX
> in
> MXNet will enable users to move between frameworks with their models, thi=
s
> will also enable MXNet project to be a part of the ONNX open standard and
> steer the direction of ONNX.
>
>
> For those who don=E2=80=99t know ONNX, ONNX is an open source format for =
AI models
> which enables models to be transferred between frameworks. Refer to
> https://github.com/onnx/onnx for more details.
>
>
> To implement the import/export functionality in MXNet, I propose to expos=
e
> a MXNet python module =E2=80=9Cserde=E2=80=9D(name taken from Apache Hive=
 project) with the
> following methods supporting different formats:
>
> sym, params =3D mxnet.serde.import(other_format_file, other_format=3D=E2=
=80=98onnx=E2=80=99)
>
> other_format_file =3D  mxnet.serde.export(mxnet_sym, mxnet_params, =E2=80=
=98onnx=E2=80=99)
>
>
> The implementation under the hood can be done in two ways:
>
>
> 1) Implement at the MXNet layer by parsing the ONNX model(in protobuf
> format) and turn into MXNet Symbolic operators and build MXNet model
> directly. Similarly, I can convert the MXNet model to ONNX format at this
> layer.
>
>
> 2) The DMLC community has released the nnvm/tvm complier and an
> intermediate representation of the models, refer:
> http://www.tvmlang.org/2017/10/06/nnvm/tvm-compiler-announcement.html
> <http://www.tvmlang.org/2017/10/06/nnvm-compiler-announcement.html>
>
> Based on the conversation on the GitHub issue
> <https://github.com/apache/incubator-mxnet/issues/8319> I opened, Mu
> mentioned that MXNet would use nnvm/tvm as the backend in the future.
>
>
> We could hook into this layer to implement the import/export functionalit=
y.
> nnvm/tvm has ONNX 0.1 version import implemented.
>
> For import,
>
>    1.
>
>    I will need to enhance nnvm/tvm=E2=80=99s importer to support ONNX 0.2
>    2.
>
>    Implement nnvm/tvm->mxnet symbolic operators.
>
> For export:
>
>
>    1.
>
>    mxnet->nnvm/tvm ( nnvm/tvm provides this implementation already)
>    2.
>
>    I will need to Implement nnvm/tvm>onnx.
>
>
> These are the pros and cons I see in the above approaches:
>
>    1.
>
>    Import/export at mxnet layer
>
> Pros:
>
>    1.
>
>    Stable APIs currently used by users.
>    2.
>
>    Larger Apache MXNet community of contributors.
>    3.
>
>    CI pipeline to catch bugs.
>    4.
>
>    Comparatively less time to implement and put it in the hands of the
>    users.
>
> Cons:
>
>    1.
>
>    In the future we may have to reimplement at the nnvm/tvm layer, in cas=
e
>    MXNet moves to the nnvm/tvm backend(assuming it will move).
>
>
>
>    1.
>
>    Import/export at nnvm/tvm layer
>
> Pros:
>
>    1.
>
>    Less engineering work in case mxnet moves to nnvm/tvm
>    2.
>
>    nnvm/tvm would become a hub to convert to different formats.
>    3.
>
>    nnvm operators are more in parity with mxnet=E2=80=99s gluon APIs this=
 could be
>    useful in case Gluon becomes the only standard that MXNet will support=
.
>
> Cons:
>
>    1.
>
>    Nascent project with few contributors
>    2.
>
>    Does not support all operators that exist in MXNet Symbolic API
>    3.
>
>    No CI Pipeline
>    4.
>
>    Current Apache MXNet project does not use nnvm/tvm backend
>    5.
>
>    mxnet->nnvm/tvm backend needs more testing and user feedback.
>
>
> Any suggestions on both of these approaches? From user's perspective, thi=
s
> will be an implementation detail that is not exposed.
>
> Thanks,
>
> Roshani
>

--94eb2c05d86677e62a055bd8b6a3--