mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Olivier <cjolivie...@gmail.com>
Subject Re: Request for suggestions- Supporting onnx in mxnet
Date Thu, 19 Oct 2017 02:15:46 GMT
Reduce code base of mxnet? By increasing scope of the dmlc modules? Is the
intent to make mxnet a thin language wrapper around a group of dmlc
modules?


On Wed, Oct 18, 2017 at 6:58 PM Tianqi Chen <tqchen@cs.washington.edu>
wrote:

> To better answer Hagay's question, I would like to dive down a bit deeper
> on the relation between MXNet, NNVM and model exchange format like ONNX.
>
> There are two major trends in deep learning systems now:
>
> - Common serializable formats, like ONNX and CoreML, that defines the model
> exchange format.
> - The in-memory graph IR for quick optimization and JIT. NNVM, Tensorflow's
> XLA falls into this category.
>
> The exchange formats are great, it only poses a layer of conversion, which
> is good for exchange. The real meat still comes from the compilation and
> JIT pipeline you have to offer. For that, we will need an in-memory IR,
> because of the cost of constructing, serialize could be high for the
> exchange formats like protobuf.  And usually, the exchange formats are
> designed in a minimalistic fashion, making it less easy to extend more
> information to support in-depth optimization like automatic quantization,
> accelerator support.
>
> The current MXNet relies on NNVM for in-memory IR manipulation but does not
> contain a compilation component that compiles to the hardware backends.
> Doing export to an exchange format and then back into NNVM run the
> compilation poses too much burden that JIT compiler could pay. Using the
> same in-memory graph IR as the compilation stack give much more advantage
> in terms of this.
>
> The newly introduces nnvm/top and compiler offers in-memory graph
> optimization and compilation and offers more hardware backend directly via
> TVM. We already see promising results in edge deployments with a much lower
> overhead of runtime. We will further benefit quickly from more graph
> optimizations that it has to offer.
>
> Building support around this new paradigm offers us advantage of being
> future compatible and takes full benefit of the points I mentioned above
>
> Tianqi
>
>
>
> On Wed, Oct 18, 2017 at 4:57 PM, Lupesko, Hagay <lupesko@gmail.com> wrote:
>
> > Roshani – this is an exciting initiative, ONNX support on MXNet will
> > enable more users to ramp up on MXNet, which is great.
> >
> > Tianqi – a few questions and thoughts about your note:
> > - “More hardware backends to mxnet” – MXNet users get the same benefit of
> > HW support implementing ONNX import on top of MXNet symbolic, right?
> > - “NNVM Compiler now received contributions from AWS, UW and many other
> > folks in MXNet community.” – agreed it is ramping up, but when you look
> at
> > the data, it is clear that it is very early on for NNVM. Looking at the
> > repo, it has overall 223 commits, 0 releases. Compare it to MXNet with
> 6136
> > commits and 32 releases. It seems to be still early on for NNVM, and for
> a
> > more reliable initial implementation building the import on top of MXNet
> is
> > easier, faster and safer. MXNet has lots of users already using the
> > Symbolic API which hopefully mean that is a mature API that is not likely
> > to have breaking changes or major issues.
> >
> > I’m supportive option 1 proposed by Roshani (building serde on top of
> > MXNet symbolic), but to do it as an encapsulated implementation detail,
> so
> > the implementation can be migrated to NNVM or another implementation in
> the
> > future, if at that point it seems like the right thing to do.
> >
> > Interested in hearing other opinions though…
> >
> > Hagay
> >
> > On 10/18/17, 14:13, "Tianqi Chen" <workcrow@gmail.com on behalf of
> > tqchen@cs.washington.edu> wrote:
> >
> >     I am strongly recommending going through the nnvm/top. One major
> > reason in
> >     here is that the support of nnvm/top layer NOT ONLY mean
> compatibility
> > of
> >     model format with onnx. These are the major benefits:
> >
> >
> >     - More hardware backends to mxnet, including opencl, metal, Raspberry
> > Pi,
> >     web browser. These things are automatically enabled by going through
> > this
> >     layer. In general, we design nnvm/tvm stack to resolve the challenge
> of
> >     current mxnet's weakness in terms deploying to more hardware
> backends.
> >
> >     - More frontend capabilities, nnvm's gluon style IR ingests now from
> >     CoreML, ONNX and in future keras. Supporting those will reduce the
> > amount
> >     of engineering effort needed.
> >
> >     - Future compatibility. We all agree that the future being migrated
> to
> >     gluon's API. NNVM/top tries to look ahead by directly adopting the
> > symbolic
> >     API to be gluon.
> >
> >
> >     I would also like to correct some of the mentioned facts with regard
> to
> >     nnvm/tvm stack
> >
> >     1.   Nascent project with few contributors
> >
> >     NNVM Compiler now received contributions from AWS, UW and many other
> > folks
> >     in MXNet community. NNVM itself is already being used by MXNet.
> >     MXNet's internal IR is migrating toward gluon, and its final form
> being
> >     nnvm/top
> >
> >     3.   Does not support all operators that exist in MXNet Symbolic API
> >
> >     Neither NNVM/top or onnx support all operators that exist in mxnet
> > symbolic
> >     API. The end goal here is mainly to make nnvm/top onnx compatible,
> > which is
> >     a more reasonable goal.
> >
> >     4.  No CI Pipeline and testcases
> >
> >     NNVM already contains a compiler contains unittests and ci tested
> with
> >     integration  https://github.com/dmlc/nnvm, with a CI pipline that is
> > well
> >     tested on CPU and GPU cases for front-ends.
> >
> >     Tianqi
> >
> >
> >     On Wed, Oct 18, 2017 at 1:41 PM, Roshani Nagmote <
> > roshaninagmote2@gmail.com>
> >     wrote:
> >
> >     > Hi guys,
> >     >
> >     >
> >     > I am working on supporting ONNX <https://github.com/onnx/onnx>
> > pre-trained
> >     > models in Apache MXNet and would like to seek your opinion on the
> > choice of
> >     > implementation. I also have created a GitHub issue
> >     > <https://github.com/apache/incubator-mxnet/issues/8319>.
> Supporting
> > ONNX
> >     > in
> >     > MXNet will enable users to move between frameworks with their
> > models, this
> >     > will also enable MXNet project to be a part of the ONNX open
> > standard and
> >     > steer the direction of ONNX.
> >     >
> >     >
> >     > For those who don’t know ONNX, ONNX is an open source format for AI
> > models
> >     > which enables models to be transferred between frameworks. Refer to
> >     > https://github.com/onnx/onnx for more details.
> >     >
> >     >
> >     > To implement the import/export functionality in MXNet, I propose to
> > expose
> >     > a MXNet python module “serde”(name taken from Apache Hive project)
> > with the
> >     > following methods supporting different formats:
> >     >
> >     > sym, params = mxnet.serde.import(other_format_file,
> > other_format=‘onnx’)
> >     >
> >     > other_format_file =  mxnet.serde.export(mxnet_sym, mxnet_params,
> > ‘onnx’)
> >     >
> >     >
> >     > The implementation under the hood can be done in two ways:
> >     >
> >     >
> >     > 1) Implement at the MXNet layer by parsing the ONNX model(in
> protobuf
> >     > format) and turn into MXNet Symbolic operators and build MXNet
> model
> >     > directly. Similarly, I can convert the MXNet model to ONNX format
> at
> > this
> >     > layer.
> >     >
> >     >
> >     > 2) The DMLC community has released the nnvm/tvm complier and an
> >     > intermediate representation of the models, refer:
> >     > http://www.tvmlang.org/2017/10/06/nnvm/tvm-compiler-
> > announcement.html
> >     > <http://www.tvmlang.org/2017/10/06/nnvm-compiler-announcement.html
> >
> >     >
> >     > Based on the conversation on the GitHub issue
> >     > <https://github.com/apache/incubator-mxnet/issues/8319> I opened,
> Mu
> >     > mentioned that MXNet would use nnvm/tvm as the backend in the
> future.
> >     >
> >     >
> >     > We could hook into this layer to implement the import/export
> > functionality.
> >     > nnvm/tvm has ONNX 0.1 version import implemented.
> >     >
> >     > For import,
> >     >
> >     >    1.
> >     >
> >     >    I will need to enhance nnvm/tvm’s importer to support ONNX 0.2
> >     >    2.
> >     >
> >     >    Implement nnvm/tvm->mxnet symbolic operators.
> >     >
> >     > For export:
> >     >
> >     >
> >     >    1.
> >     >
> >     >    mxnet->nnvm/tvm ( nnvm/tvm provides this implementation already)
> >     >    2.
> >     >
> >     >    I will need to Implement nnvm/tvm>onnx.
> >     >
> >     >
> >     > These are the pros and cons I see in the above approaches:
> >     >
> >     >    1.
> >     >
> >     >    Import/export at mxnet layer
> >     >
> >     > Pros:
> >     >
> >     >    1.
> >     >
> >     >    Stable APIs currently used by users.
> >     >    2.
> >     >
> >     >    Larger Apache MXNet community of contributors.
> >     >    3.
> >     >
> >     >    CI pipeline to catch bugs.
> >     >    4.
> >     >
> >     >    Comparatively less time to implement and put it in the hands of
> > the
> >     >    users.
> >     >
> >     > Cons:
> >     >
> >     >    1.
> >     >
> >     >    In the future we may have to reimplement at the nnvm/tvm layer,
> > in case
> >     >    MXNet moves to the nnvm/tvm backend(assuming it will move).
> >     >
> >     >
> >     >
> >     >    1.
> >     >
> >     >    Import/export at nnvm/tvm layer
> >     >
> >     > Pros:
> >     >
> >     >    1.
> >     >
> >     >    Less engineering work in case mxnet moves to nnvm/tvm
> >     >    2.
> >     >
> >     >    nnvm/tvm would become a hub to convert to different formats.
> >     >    3.
> >     >
> >     >    nnvm operators are more in parity with mxnet’s gluon APIs this
> > could be
> >     >    useful in case Gluon becomes the only standard that MXNet will
> > support.
> >     >
> >     > Cons:
> >     >
> >     >    1.
> >     >
> >     >    Nascent project with few contributors
> >     >    2.
> >     >
> >     >    Does not support all operators that exist in MXNet Symbolic API
> >     >    3.
> >     >
> >     >    No CI Pipeline
> >     >    4.
> >     >
> >     >    Current Apache MXNet project does not use nnvm/tvm backend
> >     >    5.
> >     >
> >     >    mxnet->nnvm/tvm backend needs more testing and user feedback.
> >     >
> >     >
> >     > Any suggestions on both of these approaches? From user's
> > perspective, this
> >     > will be an implementation detail that is not exposed.
> >     >
> >     > Thanks,
> >     >
> >     > Roshani
> >     >
> >
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message