Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mxnet.incubator.apache.org
MIME-Version: 1.0
References: <CAC-xMDL+9_A=LdN1BVNs7Mg28zs_hG8ULsDWBW3kTGw1KQmLKw@mail.gmail.com>
 <CACZfuxZqAHYjX90mKcYBkDXfLPdvKHJLfiwBK38xJnxme-yyDw@mail.gmail.com>
 <B0B535A1-1C73-4DCD-8CB2-66039A363A1D@gmail.com> <CACZfuxZxHRfp11pbxBj084FfLunuzZP+xuW28v-EP_Cia6fxdA@mail.gmail.com>
In-Reply-To: <CACZfuxZxHRfp11pbxBj084FfLunuzZP+xuW28v-EP_Cia6fxdA@mail.gmail.com>
From: Chris Olivier <cjolivier01@gmail.com>
Date: Thu, 19 Oct 2017 02:15:46 +0000
Message-ID: <CABgAAfe0cuA3nKYEVZZJmdfGZhG5CP5s-9T6csG5H3iE4RfimA@mail.gmail.com>
Subject: Re: Request for suggestions- Supporting onnx in mxnet
To: dev@mxnet.incubator.apache.org
Content-Type: multipart/alternative; boundary="089e0825e940c28af1055bdcef96"
archived-at: Thu, 19 Oct 2017 02:16:06 -0000

--089e0825e940c28af1055bdcef96
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Reduce code base of mxnet? By increasing scope of the dmlc modules? Is the
intent to make mxnet a thin language wrapper around a group of dmlc
modules?


On Wed, Oct 18, 2017 at 6:58 PM Tianqi Chen <tqchen@cs.washington.edu>
wrote:

> To better answer Hagay's question, I would like to dive down a bit deeper
> on the relation between MXNet, NNVM and model exchange format like ONNX.
>
> There are two major trends in deep learning systems now:
>
> - Common serializable formats, like ONNX and CoreML, that defines the mod=
el
> exchange format.
> - The in-memory graph IR for quick optimization and JIT. NNVM, Tensorflow=
's
> XLA falls into this category.
>
> The exchange formats are great, it only poses a layer of conversion, whic=
h
> is good for exchange. The real meat still comes from the compilation and
> JIT pipeline you have to offer. For that, we will need an in-memory IR,
> because of the cost of constructing, serialize could be high for the
> exchange formats like protobuf.  And usually, the exchange formats are
> designed in a minimalistic fashion, making it less easy to extend more
> information to support in-depth optimization like automatic quantization,
> accelerator support.
>
> The current MXNet relies on NNVM for in-memory IR manipulation but does n=
ot
> contain a compilation component that compiles to the hardware backends.
> Doing export to an exchange format and then back into NNVM run the
> compilation poses too much burden that JIT compiler could pay. Using the
> same in-memory graph IR as the compilation stack give much more advantage
> in terms of this.
>
> The newly introduces nnvm/top and compiler offers in-memory graph
> optimization and compilation and offers more hardware backend directly vi=
a
> TVM. We already see promising results in edge deployments with a much low=
er
> overhead of runtime. We will further benefit quickly from more graph
> optimizations that it has to offer.
>
> Building support around this new paradigm offers us advantage of being
> future compatible and takes full benefit of the points I mentioned above
>
> Tianqi
>
>
>
> On Wed, Oct 18, 2017 at 4:57 PM, Lupesko, Hagay <lupesko@gmail.com> wrote=
:
>
> > Roshani =E2=80=93 this is an exciting initiative, ONNX support on MXNet=
 will
> > enable more users to ramp up on MXNet, which is great.
> >
> > Tianqi =E2=80=93 a few questions and thoughts about your note:
> > - =E2=80=9CMore hardware backends to mxnet=E2=80=9D =E2=80=93 MXNet use=
rs get the same benefit of
> > HW support implementing ONNX import on top of MXNet symbolic, right?
> > - =E2=80=9CNNVM Compiler now received contributions from AWS, UW and ma=
ny other
> > folks in MXNet community.=E2=80=9D =E2=80=93 agreed it is ramping up, b=
ut when you look
> at
> > the data, it is clear that it is very early on for NNVM. Looking at the
> > repo, it has overall 223 commits, 0 releases. Compare it to MXNet with
> 6136
> > commits and 32 releases. It seems to be still early on for NNVM, and fo=
r
> a
> > more reliable initial implementation building the import on top of MXNe=
t
> is
> > easier, faster and safer. MXNet has lots of users already using the
> > Symbolic API which hopefully mean that is a mature API that is not like=
ly
> > to have breaking changes or major issues.
> >
> > I=E2=80=99m supportive option 1 proposed by Roshani (building serde on =
top of
> > MXNet symbolic), but to do it as an encapsulated implementation detail,
> so
> > the implementation can be migrated to NNVM or another implementation in
> the
> > future, if at that point it seems like the right thing to do.
> >
> > Interested in hearing other opinions though=E2=80=A6
> >
> > Hagay
> >
> > On 10/18/17, 14:13, "Tianqi Chen" <workcrow@gmail.com on behalf of
> > tqchen@cs.washington.edu> wrote:
> >
> >     I am strongly recommending going through the nnvm/top. One major
> > reason in
> >     here is that the support of nnvm/top layer NOT ONLY mean
> compatibility
> > of
> >     model format with onnx. These are the major benefits:
> >
> >
> >     - More hardware backends to mxnet, including opencl, metal, Raspber=
ry
> > Pi,
> >     web browser. These things are automatically enabled by going throug=
h
> > this
> >     layer. In general, we design nnvm/tvm stack to resolve the challeng=
e
> of
> >     current mxnet's weakness in terms deploying to more hardware
> backends.
> >
> >     - More frontend capabilities, nnvm's gluon style IR ingests now fro=
m
> >     CoreML, ONNX and in future keras. Supporting those will reduce the
> > amount
> >     of engineering effort needed.
> >
> >     - Future compatibility. We all agree that the future being migrated
> to
> >     gluon's API. NNVM/top tries to look ahead by directly adopting the
> > symbolic
> >     API to be gluon.
> >
> >
> >     I would also like to correct some of the mentioned facts with regar=
d
> to
> >     nnvm/tvm stack
> >
> >     1.   Nascent project with few contributors
> >
> >     NNVM Compiler now received contributions from AWS, UW and many othe=
r
> > folks
> >     in MXNet community. NNVM itself is already being used by MXNet.
> >     MXNet's internal IR is migrating toward gluon, and its final form
> being
> >     nnvm/top
> >
> >     3.   Does not support all operators that exist in MXNet Symbolic AP=
I
> >
> >     Neither NNVM/top or onnx support all operators that exist in mxnet
> > symbolic
> >     API. The end goal here is mainly to make nnvm/top onnx compatible,
> > which is
> >     a more reasonable goal.
> >
> >     4.  No CI Pipeline and testcases
> >
> >     NNVM already contains a compiler contains unittests and ci tested
> with
> >     integration  https://github.com/dmlc/nnvm, with a CI pipline that i=
s
> > well
> >     tested on CPU and GPU cases for front-ends.
> >
> >     Tianqi
> >
> >
> >     On Wed, Oct 18, 2017 at 1:41 PM, Roshani Nagmote <
> > roshaninagmote2@gmail.com>
> >     wrote:
> >
> >     > Hi guys,
> >     >
> >     >
> >     > I am working on supporting ONNX <https://github.com/onnx/onnx>
> > pre-trained
> >     > models in Apache MXNet and would like to seek your opinion on the
> > choice of
> >     > implementation. I also have created a GitHub issue
> >     > <https://github.com/apache/incubator-mxnet/issues/8319>.
> Supporting
> > ONNX
> >     > in
> >     > MXNet will enable users to move between frameworks with their
> > models, this
> >     > will also enable MXNet project to be a part of the ONNX open
> > standard and
> >     > steer the direction of ONNX.
> >     >
> >     >
> >     > For those who don=E2=80=99t know ONNX, ONNX is an open source for=
mat for AI
> > models
> >     > which enables models to be transferred between frameworks. Refer =
to
> >     > https://github.com/onnx/onnx for more details.
> >     >
> >     >
> >     > To implement the import/export functionality in MXNet, I propose =
to
> > expose
> >     > a MXNet python module =E2=80=9Cserde=E2=80=9D(name taken from Apa=
che Hive project)
> > with the
> >     > following methods supporting different formats:
> >     >
> >     > sym, params =3D mxnet.serde.import(other_format_file,
> > other_format=3D=E2=80=98onnx=E2=80=99)
> >     >
> >     > other_format_file =3D  mxnet.serde.export(mxnet_sym, mxnet_params=
,
> > =E2=80=98onnx=E2=80=99)
> >     >
> >     >
> >     > The implementation under the hood can be done in two ways:
> >     >
> >     >
> >     > 1) Implement at the MXNet layer by parsing the ONNX model(in
> protobuf
> >     > format) and turn into MXNet Symbolic operators and build MXNet
> model
> >     > directly. Similarly, I can convert the MXNet model to ONNX format
> at
> > this
> >     > layer.
> >     >
> >     >
> >     > 2) The DMLC community has released the nnvm/tvm complier and an
> >     > intermediate representation of the models, refer:
> >     > http://www.tvmlang.org/2017/10/06/nnvm/tvm-compiler-
> > announcement.html
> >     > <http://www.tvmlang.org/2017/10/06/nnvm-compiler-announcement.htm=
l
> >
> >     >
> >     > Based on the conversation on the GitHub issue
> >     > <https://github.com/apache/incubator-mxnet/issues/8319> I opened,
> Mu
> >     > mentioned that MXNet would use nnvm/tvm as the backend in the
> future.
> >     >
> >     >
> >     > We could hook into this layer to implement the import/export
> > functionality.
> >     > nnvm/tvm has ONNX 0.1 version import implemented.
> >     >
> >     > For import,
> >     >
> >     >    1.
> >     >
> >     >    I will need to enhance nnvm/tvm=E2=80=99s importer to support =
ONNX 0.2
> >     >    2.
> >     >
> >     >    Implement nnvm/tvm->mxnet symbolic operators.
> >     >
> >     > For export:
> >     >
> >     >
> >     >    1.
> >     >
> >     >    mxnet->nnvm/tvm ( nnvm/tvm provides this implementation alread=
y)
> >     >    2.
> >     >
> >     >    I will need to Implement nnvm/tvm>onnx.
> >     >
> >     >
> >     > These are the pros and cons I see in the above approaches:
> >     >
> >     >    1.
> >     >
> >     >    Import/export at mxnet layer
> >     >
> >     > Pros:
> >     >
> >     >    1.
> >     >
> >     >    Stable APIs currently used by users.
> >     >    2.
> >     >
> >     >    Larger Apache MXNet community of contributors.
> >     >    3.
> >     >
> >     >    CI pipeline to catch bugs.
> >     >    4.
> >     >
> >     >    Comparatively less time to implement and put it in the hands o=
f
> > the
> >     >    users.
> >     >
> >     > Cons:
> >     >
> >     >    1.
> >     >
> >     >    In the future we may have to reimplement at the nnvm/tvm layer=
,
> > in case
> >     >    MXNet moves to the nnvm/tvm backend(assuming it will move).
> >     >
> >     >
> >     >
> >     >    1.
> >     >
> >     >    Import/export at nnvm/tvm layer
> >     >
> >     > Pros:
> >     >
> >     >    1.
> >     >
> >     >    Less engineering work in case mxnet moves to nnvm/tvm
> >     >    2.
> >     >
> >     >    nnvm/tvm would become a hub to convert to different formats.
> >     >    3.
> >     >
> >     >    nnvm operators are more in parity with mxnet=E2=80=99s gluon A=
PIs this
> > could be
> >     >    useful in case Gluon becomes the only standard that MXNet will
> > support.
> >     >
> >     > Cons:
> >     >
> >     >    1.
> >     >
> >     >    Nascent project with few contributors
> >     >    2.
> >     >
> >     >    Does not support all operators that exist in MXNet Symbolic AP=
I
> >     >    3.
> >     >
> >     >    No CI Pipeline
> >     >    4.
> >     >
> >     >    Current Apache MXNet project does not use nnvm/tvm backend
> >     >    5.
> >     >
> >     >    mxnet->nnvm/tvm backend needs more testing and user feedback.
> >     >
> >     >
> >     > Any suggestions on both of these approaches? From user's
> > perspective, this
> >     > will be an implementation detail that is not exposed.
> >     >
> >     > Thanks,
> >     >
> >     > Roshani
> >     >
> >
> >
> >
> >
>

--089e0825e940c28af1055bdcef96--