mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianqi Chen <tqc...@cs.washington.edu>
Subject Re: The Exchange Layer Support of MXNet
Date Fri, 20 Oct 2017 16:05:31 GMT
As with every change, there need to be steps. The python API of gluon is a
first step toward moving API to the set of the new operator. I think
exchange and compilation are next step it is relatively easy to do so
without start implementing these ops (we canonicalize to core ops and
convert them back to the ops we have).   The final step is to move the
implementation toward the new ops, which can happen gradually.

Tianqi

On Fri, Oct 20, 2017 at 8:58 AM, Tianqi Chen <tqchen@cs.washington.edu>
wrote:

> The core ops do not mean implementation. They are merely a spec that we
> want to agree on what attributes we need.
>
> - The current list of the proposal spec is here:
> http://nnvm.tvmlang.org/top.html  (I think eventually MXNet's document
> should have one such page)
> - They are is modeled after numpy and mxnet/gluon and is used by the new
> graph optimizations in nnvm compiler.
> - There is priority in terms of support level, which indicates how
> important they are when support.
>
> The major need for core ops is to be concise(smallest set),
> expressive(covers most need), conventional (follows existing well adopted
> API like numpy). The legacy ops have no problem in terms of expressive but
> have issues with respect to conciseness and conventional.
>
> By drifting the central support to core ops and canonicalize other ops
> into them, we can make optimization, exchange and other related problems
> easier. It also gives clear external support of ApacheMXNet
>
> Tinaqi
>
> On Fri, Oct 20, 2017 at 8:50 AM, Chris Olivier <cjolivier01@gmail.com>
> wrote:
>
>> Thansk for that explanation!
>> What is meant by the "core ops"?  Are these the gluon python ops?  I know
>> what the "legacy ops" are -- things like BatchNormProp/BatchNormOp.  WHat
>> are the "new ones"?
>>
>> I apologize for not knowing this, but what are some of the "pain points"
>> with the legacy ops?  Not in great detail, but would just like to get a
>> little perspective.
>>
>> Thanks,
>>
>> -Chris
>>
>>
>> On Fri, Oct 20, 2017 at 8:42 AM, Tianqi Chen <tqchen@cs.washington.edu>
>> wrote:
>>
>> > First of all, I agree that supporting the format as an important step of
>> > adding influence. We are going to do it in either options. The overhead
>> of
>> > engineering is not that much difference.  I also do not argue for
>> specific
>> > types of APIs(gluon vs symbolic) we should use.
>> >
>> >  MXNet Sym API contains two elements:
>> >
>> > - 1) The graph manipulation API
>> > - 2) The operator defs
>> >
>> > The graph manipulation API has been serving us great and we should be
>> keep
>> > using it for whatever purposes we have. However, the operator
>> definitions
>> > has been involved for two years and there are quite a few pains in
>> legacy
>> > operator and attributes.  The new remodeled gluon API is much cleaner.
>> > While there is a gap between the current operator def and gluon's API,
>> we
>> > want to shift defs toward gluon style operator defs (NOTE this do not
>> mean
>> > imperative only, just to make operator def align with numpy and gluon).
>> >
>> > What I am proposing is to have a clear document and list of "core
>> operator
>> > defs" we want to prioritize in terms of supporting exchange offers
>> > optimization.  Having such document, gives us and others a clear
>> reference
>> > on what we need, and what is needed in order to give good external
>> support
>> > to ApacheMXNet.
>> >
>> > The MXNet/gluon or nnvm/top is such initial set of operator defs that is
>> > proposed. By canonicalizing the current MXNet op defs into this "core
>> set",
>> > and using it as a center for talking to other exchange format gives the
>> > advantage of the things we mentioned above.
>> >
>> > Going through the path have second major advantage, because it
>> > enables(answering your question on what compilation mean)
>> >       mxnet->core ops -> nnvm compiler-> bare metal(javascript, ARM,
>> > AMDGPU, Metal)
>> >
>> > While in theory we can also do mxnet->onnx-> nnvm compiler, This is a
>> more
>> > twisted path as the exchange format do not preserve information and
>> subject
>> > to change. While going through the core ops offers bi-directional
>> exchange
>> > with mxnet, and directly doing that in memory.  The compilation path is
>> not
>> > offered if we do not go through the core ops, because the graph
>> > optimization can take great advantage of a clear core set of operators.
>> >
>> > To facilitate discussion technically. I would suggest we break our
>> points
>> > down. I tried to do that in this email on things we agree on and have
>> > opions. So we can find common ground and move forward.
>> >
>> >
>> > Tianqi
>> >
>> >
>> > > On 10/19/17, 20:43, "Tianqi Chen" <workcrow@gmail.com on behalf of
>> > > tqchen@cs.washington.edu> wrote:
>> > >
>> > >     I will start forking the previous discussion and it has gone awry
>> > and I
>> > >     hope to start a pure technical discussion thread.
>> > >
>> > >     I said in another email that we could do a vote to settle this
>> issue.
>> > > I now
>> > >     think that I was wrong and would like to apology for my rush
>> proposal
>> > > on
>> > >     this.
>> > >
>> > >     I hope to reopen this email thread to gain consensus on what we
>> want.
>> > > There
>> > >     has been express of concerns that the code should reside on
>> > ApacheMXNet
>> > >     repo. I think that discussion is already over and at least I would
>> > > want the
>> > >     code to be in the repo as long as we reach the consensus.
>> > >
>> > >     The leftover point is how should we do it. There are two ways of
>> > doing
>> > > this
>> > >
>> > >     1) Doing it on top of existing Symbol API.
>> > >     2) Moving most of the exchange layer on standardized core operator
>> > set,
>> > >     namely mxnet/gluon spec.
>> > >
>> > >     Both approaches are feasible. There is some advocation on which
>> way
>> > > might
>> > >     be simpler, but the additional effort of engineering won't be that
>> > > much.
>> > >     The reason why I am seeing this decision so seriously is that it
>> will
>> > >     affect how we can influence the design of the exchange format we
>> act
>> > > on, and
>> > >     how easily it is to integrate future features that are made
>> > available.
>> > >
>> > >     I am in favor of (2) because technically it gives us a clean
>> future
>> > >     compatibility, offers compilation and articulates clearly what
>> > >     ApacheMXNet's stance on core operators.
>> > >     These things could bring more impact to the community in general.
>> > >
>> > >     Tianqi
>> > >
>> > >
>> > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message