Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mxnet.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CACZfuxZnwV_eBJS1GP_696oDNXOJt9k02qSpaiSuorZ+uk7t7g@mail.gmail.com>
References: <CACZfuxYaom62p7EXMVkG41b22RxDvYYvBPJP_gzw5p44x7-KwQ@mail.gmail.com>
 <915ECA45-2CA1-4E34-864B-CFF6ABEDAF33@gmail.com> <CACZfuxZnwV_eBJS1GP_696oDNXOJt9k02qSpaiSuorZ+uk7t7g@mail.gmail.com>
From: Chris Olivier <cjolivier01@gmail.com>
Date: Fri, 20 Oct 2017 08:50:42 -0700
Message-ID: <CABgAAfccshp5MYwcq+jDuoMG8TWBYDPnrCYPwzkBvkrgvkVZLg@mail.gmail.com>
Subject: Re: The Exchange Layer Support of MXNet
To: dev@mxnet.incubator.apache.org
Content-Type: multipart/alternative; boundary="001a11447da46c914d055bfc6f02"
archived-at: Fri, 20 Oct 2017 15:50:49 -0000

--001a11447da46c914d055bfc6f02
Content-Type: text/plain; charset="UTF-8"

Thansk for that explanation!
What is meant by the "core ops"?  Are these the gluon python ops?  I know
what the "legacy ops" are -- things like BatchNormProp/BatchNormOp.  WHat
are the "new ones"?

I apologize for not knowing this, but what are some of the "pain points"
with the legacy ops?  Not in great detail, but would just like to get a
little perspective.

Thanks,

-Chris


On Fri, Oct 20, 2017 at 8:42 AM, Tianqi Chen <tqchen@cs.washington.edu>
wrote:

> First of all, I agree that supporting the format as an important step of
> adding influence. We are going to do it in either options. The overhead of
> engineering is not that much difference.  I also do not argue for specific
> types of APIs(gluon vs symbolic) we should use.
>
>  MXNet Sym API contains two elements:
>
> - 1) The graph manipulation API
> - 2) The operator defs
>
> The graph manipulation API has been serving us great and we should be keep
> using it for whatever purposes we have. However, the operator definitions
> has been involved for two years and there are quite a few pains in legacy
> operator and attributes.  The new remodeled gluon API is much cleaner.
> While there is a gap between the current operator def and gluon's API, we
> want to shift defs toward gluon style operator defs (NOTE this do not mean
> imperative only, just to make operator def align with numpy and gluon).
>
> What I am proposing is to have a clear document and list of "core operator
> defs" we want to prioritize in terms of supporting exchange offers
> optimization.  Having such document, gives us and others a clear reference
> on what we need, and what is needed in order to give good external support
> to ApacheMXNet.
>
> The MXNet/gluon or nnvm/top is such initial set of operator defs that is
> proposed. By canonicalizing the current MXNet op defs into this "core set",
> and using it as a center for talking to other exchange format gives the
> advantage of the things we mentioned above.
>
> Going through the path have second major advantage, because it
> enables(answering your question on what compilation mean)
>       mxnet->core ops -> nnvm compiler-> bare metal(javascript, ARM,
> AMDGPU, Metal)
>
> While in theory we can also do mxnet->onnx-> nnvm compiler, This is a more
> twisted path as the exchange format do not preserve information and subject
> to change. While going through the core ops offers bi-directional exchange
> with mxnet, and directly doing that in memory.  The compilation path is not
> offered if we do not go through the core ops, because the graph
> optimization can take great advantage of a clear core set of operators.
>
> To facilitate discussion technically. I would suggest we break our points
> down. I tried to do that in this email on things we agree on and have
> opions. So we can find common ground and move forward.
>
>
> Tianqi
>
>
> > On 10/19/17, 20:43, "Tianqi Chen" <workcrow@gmail.com on behalf of
> > tqchen@cs.washington.edu> wrote:
> >
> >     I will start forking the previous discussion and it has gone awry
> and I
> >     hope to start a pure technical discussion thread.
> >
> >     I said in another email that we could do a vote to settle this issue.
> > I now
> >     think that I was wrong and would like to apology for my rush proposal
> > on
> >     this.
> >
> >     I hope to reopen this email thread to gain consensus on what we want.
> > There
> >     has been express of concerns that the code should reside on
> ApacheMXNet
> >     repo. I think that discussion is already over and at least I would
> > want the
> >     code to be in the repo as long as we reach the consensus.
> >
> >     The leftover point is how should we do it. There are two ways of
> doing
> > this
> >
> >     1) Doing it on top of existing Symbol API.
> >     2) Moving most of the exchange layer on standardized core operator
> set,
> >     namely mxnet/gluon spec.
> >
> >     Both approaches are feasible. There is some advocation on which way
> > might
> >     be simpler, but the additional effort of engineering won't be that
> > much.
> >     The reason why I am seeing this decision so seriously is that it will
> >     affect how we can influence the design of the exchange format we act
> > on, and
> >     how easily it is to integrate future features that are made
> available.
> >
> >     I am in favor of (2) because technically it gives us a clean future
> >     compatibility, offers compilation and articulates clearly what
> >     ApacheMXNet's stance on core operators.
> >     These things could bring more impact to the community in general.
> >
> >     Tianqi
> >
> >
> >
> >
>

--001a11447da46c914d055bfc6f02--