mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Indhu <indhubhara...@gmail.com>
Subject Re: The Exchange Layer Support of MXNet
Date Mon, 23 Oct 2017 20:30:33 GMT
Do we have a consensus yet on how to implement this?


On Fri, Oct 20, 2017 at 9:05 AM Tianqi Chen <tqchen@cs.washington.edu>
wrote:

> As with every change, there need to be steps. The python API of gluon is a
> first step toward moving API to the set of the new operator. I think
> exchange and compilation are next step it is relatively easy to do so
> without start implementing these ops (we canonicalize to core ops and
> convert them back to the ops we have).   The final step is to move the
> implementation toward the new ops, which can happen gradually.
>
> Tianqi
>
> On Fri, Oct 20, 2017 at 8:58 AM, Tianqi Chen <tqchen@cs.washington.edu>
> wrote:
>
> > The core ops do not mean implementation. They are merely a spec that we
> > want to agree on what attributes we need.
> >
> > - The current list of the proposal spec is here:
> > http://nnvm.tvmlang.org/top.html  (I think eventually MXNet's document
> > should have one such page)
> > - They are is modeled after numpy and mxnet/gluon and is used by the new
> > graph optimizations in nnvm compiler.
> > - There is priority in terms of support level, which indicates how
> > important they are when support.
> >
> > The major need for core ops is to be concise(smallest set),
> > expressive(covers most need), conventional (follows existing well adopted
> > API like numpy). The legacy ops have no problem in terms of expressive
> but
> > have issues with respect to conciseness and conventional.
> >
> > By drifting the central support to core ops and canonicalize other ops
> > into them, we can make optimization, exchange and other related problems
> > easier. It also gives clear external support of ApacheMXNet
> >
> > Tinaqi
> >
> > On Fri, Oct 20, 2017 at 8:50 AM, Chris Olivier <cjolivier01@gmail.com>
> > wrote:
> >
> >> Thansk for that explanation!
> >> What is meant by the "core ops"?  Are these the gluon python ops?  I
> know
> >> what the "legacy ops" are -- things like BatchNormProp/BatchNormOp.
> WHat
> >> are the "new ones"?
> >>
> >> I apologize for not knowing this, but what are some of the "pain points"
> >> with the legacy ops?  Not in great detail, but would just like to get a
> >> little perspective.
> >>
> >> Thanks,
> >>
> >> -Chris
> >>
> >>
> >> On Fri, Oct 20, 2017 at 8:42 AM, Tianqi Chen <tqchen@cs.washington.edu>
> >> wrote:
> >>
> >> > First of all, I agree that supporting the format as an important step
> of
> >> > adding influence. We are going to do it in either options. The
> overhead
> >> of
> >> > engineering is not that much difference.  I also do not argue for
> >> specific
> >> > types of APIs(gluon vs symbolic) we should use.
> >> >
> >> >  MXNet Sym API contains two elements:
> >> >
> >> > - 1) The graph manipulation API
> >> > - 2) The operator defs
> >> >
> >> > The graph manipulation API has been serving us great and we should be
> >> keep
> >> > using it for whatever purposes we have. However, the operator
> >> definitions
> >> > has been involved for two years and there are quite a few pains in
> >> legacy
> >> > operator and attributes.  The new remodeled gluon API is much cleaner.
> >> > While there is a gap between the current operator def and gluon's API,
> >> we
> >> > want to shift defs toward gluon style operator defs (NOTE this do not
> >> mean
> >> > imperative only, just to make operator def align with numpy and
> gluon).
> >> >
> >> > What I am proposing is to have a clear document and list of "core
> >> operator
> >> > defs" we want to prioritize in terms of supporting exchange offers
> >> > optimization.  Having such document, gives us and others a clear
> >> reference
> >> > on what we need, and what is needed in order to give good external
> >> support
> >> > to ApacheMXNet.
> >> >
> >> > The MXNet/gluon or nnvm/top is such initial set of operator defs that
> is
> >> > proposed. By canonicalizing the current MXNet op defs into this "core
> >> set",
> >> > and using it as a center for talking to other exchange format gives
> the
> >> > advantage of the things we mentioned above.
> >> >
> >> > Going through the path have second major advantage, because it
> >> > enables(answering your question on what compilation mean)
> >> >       mxnet->core ops -> nnvm compiler-> bare metal(javascript,
ARM,
> >> > AMDGPU, Metal)
> >> >
> >> > While in theory we can also do mxnet->onnx-> nnvm compiler, This
is a
> >> more
> >> > twisted path as the exchange format do not preserve information and
> >> subject
> >> > to change. While going through the core ops offers bi-directional
> >> exchange
> >> > with mxnet, and directly doing that in memory.  The compilation path
> is
> >> not
> >> > offered if we do not go through the core ops, because the graph
> >> > optimization can take great advantage of a clear core set of
> operators.
> >> >
> >> > To facilitate discussion technically. I would suggest we break our
> >> points
> >> > down. I tried to do that in this email on things we agree on and have
> >> > opions. So we can find common ground and move forward.
> >> >
> >> >
> >> > Tianqi
> >> >
> >> >
> >> > > On 10/19/17, 20:43, "Tianqi Chen" <workcrow@gmail.com on behalf
of
> >> > > tqchen@cs.washington.edu> wrote:
> >> > >
> >> > >     I will start forking the previous discussion and it has gone
> awry
> >> > and I
> >> > >     hope to start a pure technical discussion thread.
> >> > >
> >> > >     I said in another email that we could do a vote to settle this
> >> issue.
> >> > > I now
> >> > >     think that I was wrong and would like to apology for my rush
> >> proposal
> >> > > on
> >> > >     this.
> >> > >
> >> > >     I hope to reopen this email thread to gain consensus on what we
> >> want.
> >> > > There
> >> > >     has been express of concerns that the code should reside on
> >> > ApacheMXNet
> >> > >     repo. I think that discussion is already over and at least I
> would
> >> > > want the
> >> > >     code to be in the repo as long as we reach the consensus.
> >> > >
> >> > >     The leftover point is how should we do it. There are two ways
of
> >> > doing
> >> > > this
> >> > >
> >> > >     1) Doing it on top of existing Symbol API.
> >> > >     2) Moving most of the exchange layer on standardized core
> operator
> >> > set,
> >> > >     namely mxnet/gluon spec.
> >> > >
> >> > >     Both approaches are feasible. There is some advocation on which
> >> way
> >> > > might
> >> > >     be simpler, but the additional effort of engineering won't be
> that
> >> > > much.
> >> > >     The reason why I am seeing this decision so seriously is that
it
> >> will
> >> > >     affect how we can influence the design of the exchange format
we
> >> act
> >> > > on, and
> >> > >     how easily it is to integrate future features that are made
> >> > available.
> >> > >
> >> > >     I am in favor of (2) because technically it gives us a clean
> >> future
> >> > >     compatibility, offers compilation and articulates clearly what
> >> > >     ApacheMXNet's stance on core operators.
> >> > >     These things could bring more impact to the community in
> general.
> >> > >
> >> > >     Tianqi
> >> > >
> >> > >
> >> > >
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message