Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mxnet.incubator.apache.org
User-Agent: Microsoft-MacOutlook/f.27.0.171010
Date: Wed, 18 Oct 2017 16:57:48 -0700
Subject: Re: Request for suggestions- Supporting onnx in mxnet
From: "Lupesko, Hagay" <lupesko@gmail.com>
To: <dev@mxnet.incubator.apache.org>
Message-ID: <B0B535A1-1C73-4DCD-8CB2-66039A363A1D@gmail.com>
Thread-Topic: Request for suggestions- Supporting onnx in mxnet
References: <CAC-xMDL+9_A=LdN1BVNs7Mg28zs_hG8ULsDWBW3kTGw1KQmLKw@mail.gmail.com>
 <CACZfuxZqAHYjX90mKcYBkDXfLPdvKHJLfiwBK38xJnxme-yyDw@mail.gmail.com>
In-Reply-To: <CACZfuxZqAHYjX90mKcYBkDXfLPdvKHJLfiwBK38xJnxme-yyDw@mail.gmail.com>
Mime-version: 1.0
Content-type: text/plain;
	charset="UTF-8"
Content-transfer-encoding: quoted-printable
archived-at: Wed, 18 Oct 2017 23:58:00 -0000

Roshani =E2=80=93 this is an exciting initiative, ONNX support on MXNet will enab=
le more users to ramp up on MXNet, which is great.

Tianqi =E2=80=93 a few questions and thoughts about your note:
- =E2=80=9CMore hardware backends to mxnet=E2=80=9D =E2=80=93 MXNet users get the same benefi=
t of HW support implementing ONNX import on top of MXNet symbolic, right?
- =E2=80=9CNNVM Compiler now received contributions from AWS, UW and many other f=
olks in MXNet community.=E2=80=9D =E2=80=93 agreed it is ramping up, but when you look a=
t the data, it is clear that it is very early on for NNVM. Looking at the re=
po, it has overall 223 commits, 0 releases. Compare it to MXNet with 6136 co=
mmits and 32 releases. It seems to be still early on for NNVM, and for a mor=
e reliable initial implementation building the import on top of MXNet is eas=
ier, faster and safer. MXNet has lots of users already using the Symbolic AP=
I which hopefully mean that is a mature API that is not likely to have break=
ing changes or major issues.

I=E2=80=99m supportive option 1 proposed by Roshani (building serde on top of MXN=
et symbolic), but to do it as an encapsulated implementation detail, so the =
implementation can be migrated to NNVM or another implementation in the futu=
re, if at that point it seems like the right thing to do.

Interested in hearing other opinions though=E2=80=A6

Hagay

On 10/18/17, 14:13, "Tianqi Chen" <workcrow@gmail.com on behalf of tqchen@c=
s.washington.edu> wrote:

    I am strongly recommending going through the nnvm/top. One major reason=
 in
    here is that the support of nnvm/top layer NOT ONLY mean compatibility =
of
    model format with onnx. These are the major benefits:
   =20
   =20
    - More hardware backends to mxnet, including opencl, metal, Raspberry P=
i,
    web browser. These things are automatically enabled by going through th=
is
    layer. In general, we design nnvm/tvm stack to resolve the challenge of
    current mxnet's weakness in terms deploying to more hardware backends.
   =20
    - More frontend capabilities, nnvm's gluon style IR ingests now from
    CoreML, ONNX and in future keras. Supporting those will reduce the amou=
nt
    of engineering effort needed.
   =20
    - Future compatibility. We all agree that the future being migrated to
    gluon's API. NNVM/top tries to look ahead by directly adopting the symb=
olic
    API to be gluon.
   =20
   =20
    I would also like to correct some of the mentioned facts with regard to
    nnvm/tvm stack
   =20
    1.   Nascent project with few contributors
   =20
    NNVM Compiler now received contributions from AWS, UW and many other fo=
lks
    in MXNet community. NNVM itself is already being used by MXNet.
    MXNet's internal IR is migrating toward gluon, and its final form being
    nnvm/top
   =20
    3.   Does not support all operators that exist in MXNet Symbolic API
   =20
    Neither NNVM/top or onnx support all operators that exist in mxnet symb=
olic
    API. The end goal here is mainly to make nnvm/top onnx compatible, whic=
h is
    a more reasonable goal.
   =20
    4.  No CI Pipeline and testcases
   =20
    NNVM already contains a compiler contains unittests and ci tested with
    integration  https://github.com/dmlc/nnvm, with a CI pipline that is we=
ll
    tested on CPU and GPU cases for front-ends.
   =20
    Tianqi
   =20
   =20
    On Wed, Oct 18, 2017 at 1:41 PM, Roshani Nagmote <roshaninagmote2@gmail=
.com>
    wrote:
   =20
    > Hi guys,
    >
    >
    > I am working on supporting ONNX <https://github.com/onnx/onnx> pre-tr=
ained
    > models in Apache MXNet and would like to seek your opinion on the cho=
ice of
    > implementation. I also have created a GitHub issue
    > <https://github.com/apache/incubator-mxnet/issues/8319>. Supporting O=
NNX
    > in
    > MXNet will enable users to move between frameworks with their models,=
 this
    > will also enable MXNet project to be a part of the ONNX open standard=
 and
    > steer the direction of ONNX.
    >
    >
    > For those who don=E2=80=99t know ONNX, ONNX is an open source format for AI=
 models
    > which enables models to be transferred between frameworks. Refer to
    > https://github.com/onnx/onnx for more details.
    >
    >
    > To implement the import/export functionality in MXNet, I propose to e=
xpose
    > a MXNet python module =E2=80=9Cserde=E2=80=9D(name taken from Apache Hive project=
) with the
    > following methods supporting different formats:
    >
    > sym, params =3D mxnet.serde.import(other_format_file, other_format=3D=E2=80=98o=
nnx=E2=80=99)
    >
    > other_format_file =3D  mxnet.serde.export(mxnet_sym, mxnet_params, =E2=80=98o=
nnx=E2=80=99)
    >
    >
    > The implementation under the hood can be done in two ways:
    >
    >
    > 1) Implement at the MXNet layer by parsing the ONNX model(in protobuf
    > format) and turn into MXNet Symbolic operators and build MXNet model
    > directly. Similarly, I can convert the MXNet model to ONNX format at =
this
    > layer.
    >
    >
    > 2) The DMLC community has released the nnvm/tvm complier and an
    > intermediate representation of the models, refer:
    > http://www.tvmlang.org/2017/10/06/nnvm/tvm-compiler-announcement.html
    > <http://www.tvmlang.org/2017/10/06/nnvm-compiler-announcement.html>
    >
    > Based on the conversation on the GitHub issue
    > <https://github.com/apache/incubator-mxnet/issues/8319> I opened, Mu
    > mentioned that MXNet would use nnvm/tvm as the backend in the future.
    >
    >
    > We could hook into this layer to implement the import/export function=
ality.
    > nnvm/tvm has ONNX 0.1 version import implemented.
    >
    > For import,
    >
    >    1.
    >
    >    I will need to enhance nnvm/tvm=E2=80=99s importer to support ONNX 0.2
    >    2.
    >
    >    Implement nnvm/tvm->mxnet symbolic operators.
    >
    > For export:
    >
    >
    >    1.
    >
    >    mxnet->nnvm/tvm ( nnvm/tvm provides this implementation already)
    >    2.
    >
    >    I will need to Implement nnvm/tvm>onnx.
    >
    >
    > These are the pros and cons I see in the above approaches:
    >
    >    1.
    >
    >    Import/export at mxnet layer
    >
    > Pros:
    >
    >    1.
    >
    >    Stable APIs currently used by users.
    >    2.
    >
    >    Larger Apache MXNet community of contributors.
    >    3.
    >
    >    CI pipeline to catch bugs.
    >    4.
    >
    >    Comparatively less time to implement and put it in the hands of th=
e
    >    users.
    >
    > Cons:
    >
    >    1.
    >
    >    In the future we may have to reimplement at the nnvm/tvm layer, in=
 case
    >    MXNet moves to the nnvm/tvm backend(assuming it will move).
    >
    >
    >
    >    1.
    >
    >    Import/export at nnvm/tvm layer
    >
    > Pros:
    >
    >    1.
    >
    >    Less engineering work in case mxnet moves to nnvm/tvm
    >    2.
    >
    >    nnvm/tvm would become a hub to convert to different formats.
    >    3.
    >
    >    nnvm operators are more in parity with mxnet=E2=80=99s gluon APIs this c=
ould be
    >    useful in case Gluon becomes the only standard that MXNet will sup=
port.
    >
    > Cons:
    >
    >    1.
    >
    >    Nascent project with few contributors
    >    2.
    >
    >    Does not support all operators that exist in MXNet Symbolic API
    >    3.
    >
    >    No CI Pipeline
    >    4.
    >
    >    Current Apache MXNet project does not use nnvm/tvm backend
    >    5.
    >
    >    mxnet->nnvm/tvm backend needs more testing and user feedback.
    >
    >
    > Any suggestions on both of these approaches? From user's perspective,=
 this
    > will be an implementation detail that is not exposed.
    >
    > Thanks,
    >
    > Roshani
    >
   =20