mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianqi Chen <tqc...@cs.washington.edu>
Subject Re: [Proposal] New operator graph for MXNet
Date Wed, 15 May 2019 00:38:59 GMT
Thanks for the proposal. Let me share some of my thoughts:

Specific comments on the proposal
-----------------------------------------------
The heavy use of generic in the Graph type was a huge departure from
type-erased data structure which was presented in the previous design.
While we understand the advantage of typed language(more compile-time
checking) and type-erased types(more dynamism) the heavy use of
the template will actually make the project solely C++ focused, making it
hard to expose intermediate(templatized) data structure to
other languages like python/scala/clojure.

While I fully understand some of the lessons taught in programming
C++(reduce shared_ptr, more typing etc.)
We need to think about the context of MXNet project and **the need to
support multi-language as a first-class**.
Some of the type-erased types are design trade-offs made to support these
features, and we need to think more
carefully instead of just applying "rules for C++" which may bring problems.

Future of NNVM
----------------------
Given that this thread touched upon what we should do for better
computational graph handling. I would recommend also to take a look at
NNVMv2 -- relay.

Relay addresses many of the wish-lists in the proposal already, such as
operator fusion, high order gradient, offload to hardware, isolated
compilation, deployment on edge and accelerators etc.
Relay also address problems not yet being mentioned in the proposal,
including control flow and dynamic runtime, automatic layout optimization
etc.

Tianqi

On Tue, May 14, 2019 at 5:06 PM Sheng Zha <zhasheng@apache.org> wrote:

> Hi Pedro,
>
> Thanks for taking the inititaive. Skimming through the design doc, I
> didn't see comparison with existing solutions such as relay in tvm, which
> is already a dependency of mxnet already. Could you elaborate on comparison
> with existing solutions in the design doc too?
>
> -sz
>
> On 2019/05/14 23:49:30, Pedro Larroy <pedro.larroy.lists@gmail.com>
> wrote:
> > Hi dev@
> >
> > As a result of my deep dives on the graph machinery I have created a
> > new proposal to improve the operator graph in MXNet.
> >
> > This would mean superseding the use of NNVM Graph in MXNet and having
> > a new implementation that we can use to simplify a lot of code and do
> > powerful graph manipulation and passes such as operator fusion and
> > other optimizations.
> >
> > As it would be a change with big impact and ramifications, your
> > thoughts and feedback on the document would be highly appreciated so
> > we can take potential future interesting use cases:
> >
> >
> https://cwiki.apache.org/confluence/display/MXNET/MXVM%3A+Operator+graph+2.0
> >
> > Pedro.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message