mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunanmcg...@gmail.com>
Subject Re: [VOTE] Disconnect all non-C API's from mxnet versioning
Date Tue, 13 Mar 2018 14:20:37 GMT
-1, I cannot think about a significant benefit comparing to the headache of
the user

still take Spark as an example

if PySpark is in a separated repo, say now it's 1.0, can you tell me
whether this 1.0 support Arrow integration without going to their website
and carefully check which version of Spark core is compiled with this 1.0?

Even with the same release cycle, I have to remember ok, 1.0 is released at
the same time with 2.3, so 1.0 must support Arrow....why I need to remember
an additional number and the association between another number and itself?

the only benefit is that, if I want to break anything, I can jump 1.0 to
2.0 directly to make it feel better, the issue is back to the cost as
aforementioned

so, -1 (binding) (doesn't matter actually, many binding -1s there)

On Mon, Mar 12, 2018 at 4:30 PM, Anirudh <anirudh2290@gmail.com> wrote:

> -1
> Because of the customer pain-points mentioned by others.
> I think for good customer experience,
> all code in MXNet repository excluding submodules and 3rd party
> dependencies
> should map to the same version.
>
> Anirudh
>
> On Mon, Mar 12, 2018 at 4:17 PM, YiZhi Liu <eazhi.liu@gmail.com> wrote:
>
> > Kellen, we are not talking about using wrong native package AFTER
> > downloading the package. It's about deciding which version to use
> > BEFORE downloading, and collecting information to debug.
> >
> > Copy-paste my previous words,
> >
> > "
> > 1. It is harmful to user experience
> >     1) Each time users want to use a specific feature, need to first
> > check the mxnet core version, then check which frontend work with this
> > core version.
> >     2) Each time users have problem using a frontend (Scala/R/...)
> > API, need to figure out which core version they are using.
> >
> > And as I describe in my #5. Imagine an inverse situation. When someone
> > has a model trained by gluon 1.6.0, he want to deploy it to JVM, what
> > Scala API version should he use? 1.6.0? No. And which R package
> > version he should use? It is still different from either Gluon version
> > or Scala API version. What a nightmare.
> > "
> >
> > 2018-03-12 16:10 GMT-07:00 kellen sunderland <
> kellen.sunderland@gmail.com
> > >:
> > > @Rahul + Roshani, I would hear what you're saying if the user had to
> > worry
> > > about using the native package, but that worry is abstracted from them.
> > > The scala package has a dependency on the native library and includes
> the
> > > native lib inside the jar.  The correct lib is then bound against at
> > > runtime.  I don't see how a user can use the wrong library or be
> confused
> > > here unless the instructions on this page are incorrect:
> > > https://github.com/apache/incubator-mxnet/tree/master/scala-package
> > >
> > > On Tue, Mar 13, 2018 at 12:02 AM, Rahul Huilgol <
> rahulhuilgol@gmail.com>
> > > wrote:
> > >
> > >> -1 for the frontends having different versions than the backend. It
> not
> > >> only creates confusion for new users, but also increases the work of
> > >> developers who need to ensure compatibility. All this for a one-time
> > change
> > >> of namespace of a package?
> > >>
> > >> I think we should increase the major version number to make this
> change?
> > >> Why do we have to 'wait' for 2.0? Who tells us that it's time for a
> 2.0
> > >> version?
> > >>
> > >> I think expecting a user to look up version numbers on the website and
> > >> ensure compatibility as suggested above, is not a simple task. Most
> > users
> > >> might not even know how the backend and frontend integrate. They might
> > not
> > >> even know that there is a C API which powers the frontends. Even
> knowing
> > >> how to look up documentation for a particular version of MXNet is a
> > >> non-trivial task right now. (And there are pages in a version's
> > >> documentation which link to a file in another version). We should
> avoid
> > >> introducing more complexity into the process. As developers we tend to
> > >> overlook the important aspect of user experience. I think we should
> > take a
> > >> step back and look at this from the perspective of a user, not from
> > that of
> > >> a developer who works closely with MXNet.
> > >>
> > >> Regards,
> > >> Rahul
> > >>
> > >> On Mon, Mar 12, 2018 at 3:12 PM, Roshani Nagmote <
> > >> roshaninagmote2@gmail.com>
> > >> wrote:
> > >>
> > >> > -1 for different versioning.
> > >> >
> > >> > I feel its just added confusion for users.
> > >> >
> > >> > On Mon, Mar 12, 2018 at 2:35 PM, YiZhi Liu <eazhi.liu@gmail.com>
> > wrote:
> > >> >
> > >> > > Agree.
> > >> > >
> > >> > > And my reply to Marco's point,
> > >> > >
> > >> > > > Changing namespaces is one use-case, but there will be a
lot
> more
> > >> with
> > >> > > increasing activity - we have to take the bigger picture in mind.
> > >> > > And you mentioned the CPP package as an example.
> > >> > > > During analysis, we figured that a re-engineering of that
API
> > would
> > >> be
> > >> > > more appropriate and easier maintainable.
> > >> > > I cannot agree as an engineer. Why not keep old API and add new
> > ones?
> > >> > > Just like in c_api.h, we added xxxEx while did not remove xxx,
> which
> > >> > > keeps APIs compatible.
> > >> > >
> > >> > > > I think it is safe to say that the other APIs have not been
> > >> maintained
> > >> > > as actively as our Python/Gluon API.
> > >> > > Are you saying, if an API is maintained actively and is widely
> used,
> > >> > > then it should be versioned together with MXNet Core?
> > >> > > Interesting, maybe instead we should have another discussion
to
> > decide
> > >> > > whether to remove some of the 'inactive' frontend bindings from
> the
> > >> > > repo.
> > >> > >
> > >> > > > We have to do #3 anyways, so it is just about having a different
> > >> number
> > >> > > set as version string.
> > >> > > A release with 6 different versions and 5 mappings?
> > >> > >
> > >> > > > I really don't see an issue in #1 - it's a simple lookup
that
> > could
> > >> be
> > >> > > done on our website.
> > >> > > Please be careful to say 'simple', each time we introduce an
> > >> > > additional step, we lose a number of our potential users.
> > >> > > And as I describe in my #5. Imagine an inverse situation. When
> > someone
> > >> > > has a model trained by gluon 1.6.0, he want to deploy it to JVM,
> > what
> > >> > > Scala API version should he use? 1.6.0? No. And which R package
> > >> > > version he should use? It is still different from either Gluon
> > version
> > >> > > or Scala API version. What a nightmare.
> > >> > >
> > >> > > 2018-03-12 14:11 GMT-07:00 Chris Olivier <cjolivier01@gmail.com>:
> > >> > > > Marco, you're mixing votes again.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > * This leaves us with three options: 1. Vote failed: No
> > refactoring
> > >> of
> > >> > > > user-facing APIs (including namespace changes) possible
OR major
> > >> > version
> > >> > > > increase 2. Vote succeeded: Refactoring of user-facing APIs
> > possible
> > >> > and
> > >> > > > only users of the changed APIs are affected while major
version
> > does
> > >> > not
> > >> > > > increase for other APIs. 3. Remove SemVer: We could introduce
> > >> breaking
> > >> > > > changes at any point in time, but our users would be losing
> trust
> > due
> > >> > to
> > >> > > > unexpected failures during upgrades.*
> > >> > > >
> > >> > > > What you're describing is not what this vote is about. 
This
> vote
> > is
> > >> > > > whether to separate mxnet and API versioning.
> > >> > > > Please try to stay on task.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Mon, Mar 12, 2018 at 1:56 PM, Marco de Abreu <
> > >> > > > marco.g.abreu@googlemail.com> wrote:
> > >> > > >
> > >> > > >> Regarding #4: Changing namespaces is one use-case, but
there
> will
> > >> be a
> > >> > > lot
> > >> > > >> more with increasing activity - we have to take the
bigger
> > picture
> > >> in
> > >> > > mind.
> > >> > > >> I think it is safe to say that the other APIs have not
been
> > >> maintained
> > >> > > as
> > >> > > >> actively as our Python/Gluon API (which I would say
could be
> > >> versioned
> > >> > > >> together with MXNet Core, but it does not really make
a
> > difference).
> > >> > > This
> > >> > > >> results in our APIs not reflecting all features available
in
> > MXNet
> > >> > (#2)
> > >> > > or
> > >> > > >> doing it in a way that we wouldn't recommend nowadays.
While it
> > is
> > >> no
> > >> > > >> problem to add new features to an API using a minor
version
> > change,
> > >> it
> > >> > > >> limits our possibilites to do a refactor. Our team,
for
> example,
> > >> got a
> > >> > > >> customer that would like to see the functionality of
the Cpp
> > package
> > >> > > being
> > >> > > >> increased. During analysis, we figured that a re-engineering
of
> > that
> > >> > API
> > >> > > >> would be more appropriate and easier maintainable. If
we don't
> > pass
> > >> > this
> > >> > > >> vote, we won't be able to make any improvements to our
less
> > >> maintained
> > >> > > APIs
> > >> > > >> without a major version increment - which the community
is also
> > >> > heavily
> > >> > > >> against. We have to do #3 anyways, so it is just about
having a
> > >> > > different
> > >> > > >> number set as version string - right now we're making
it easy
> for
> > >> > > ourselves
> > >> > > >> by basically not maintaining any other than the Python
> interface
> > and
> > >> > > >> declining all breaking changes or refactors to APIs.
I really
> > don't
> > >> > see
> > >> > > an
> > >> > > >> issue in #1 - it's a simple lookup that could be done
on our
> > >> website.
> > >> > > >> Simply select the version of MXNet you would like to
have and
> it
> > >> will
> > >> > > >> provide you with the appropriate installation instructions
-
> the
> > >> same
> > >> > > way
> > >> > > >> we're already doing it.
> > >> > > >>
> > >> > > >> This leaves us with three options:
> > >> > > >> 1. Vote failed: No refactoring of user-facing APIs (including
> > >> > namespace
> > >> > > >> changes) possible OR major version increase
> > >> > > >> 2. Vote succeeded: Refactoring of user-facing APIs possible
and
> > only
> > >> > > users
> > >> > > >> of the changed APIs are affected while major version
does not
> > >> increase
> > >> > > for
> > >> > > >> other APIs.
> > >> > > >> 3. Remove SemVer: We could introduce breaking changes
at any
> > point
> > >> in
> > >> > > time,
> > >> > > >> but our users would be losing trust due to unexpected
failures
> > >> during
> > >> > > >> upgrades.
> > >> > > >>
> > >> > > >> -Marco
> > >> > > >>
> > >> > > >>
> > >> > > >> On Mon, Mar 12, 2018 at 9:22 PM, YiZhi Liu <
> eazhi.liu@gmail.com>
> > >> > wrote:
> > >> > > >>
> > >> > > >> > STRONGLY -1 (binding) as I explained in another
thread
> > 'Publishing
> > >> > > >> > Scala Package/namespace change'. I think separating
version
> is
> > >> > > >> > harmful.
> > >> > > >> >
> > >> > > >> > 1. It is harmful to user experience
> > >> > > >> >     1) Each time users want to use a specific feature,
need
> to
> > >> first
> > >> > > >> > check the mxnet core version, then check which
frontend work
> > with
> > >> > this
> > >> > > >> > core version.
> > >> > > >> >     2) Each time users have problem using a frontend
> > (Scala/R/...)
> > >> > > >> > API, need to figure out which core version they
are using.
> > >> > > >> > 2. Frontend APIs are tightly binding to the 'MXNet
Core',
> e.g.,
> > >> > almost
> > >> > > >> > all APIs extract operator definitions from the
Core, which
> > makes
> > >> the
> > >> > > >> > API version and Core version a one-on-one mapping.
Then why
> > >> > separate?
> > >> > > >> > 3. It introduces overhead for release. Now each
release need
> to
> > >> > > >> > involve a bunch of frontend release version, along
with the
> > MXNet
> > >> > core
> > >> > > >> > release version.
> > >> > > >> > 4. The only benefit I see so far is, it makes easier
for
> Scala
> > >> > package
> > >> > > >> > to change the namespace from ml.dmlc to org.apache
(by
> > increasing
> > >> > > >> > Scala API major version id without changing the
MXNet core
> > major
> > >> > > >> > version). But,
> > >> > > >> >    1) It is very likely that, this is the ONLY
time we
> benefit
> > >> from
> > >> > > >> > separate versioning. Changing namespace is a very
rare issue
> > that
> > >> > > >> > forces us to make APIs incompatible. In other situations,
the
> > APIs
> > >> > > >> > evolves smoothly which can stay compatible for
a long time.
> > >> > > >> >    2) We can still discuss whether we have to change
the
> major
> > >> > > version.
> > >> > > >> >    3) Even the answer to 2) is Yes, I think it
is affordable
> to
> > >> wait
> > >> > > >> > for MXNet 2.0 to change 'ml.dmlc' to 'org.apache'
> > >> > > >> > 5. Other Apache projects, e.g., Apache Spark, have
PySpark
> > (Python
> > >> > > >> > frontend API), SparkR (R frontend API), MLLib,
GraphX, etc,
> > same
> > >> > > >> > version as the Spark Core, as well as the Scala/Java
API. I
> > feel
> > >> it
> > >> > > >> > convenient since every time I check a document,
say, MLLib
> > 1.6.0,
> > >> I
> > >> > > >> > can tell it works with Spark Core 1.6.0 and GraphX
1.6.0.
> And I
> > >> can
> > >> > > >> > expect when I use Python API 1.6.0, it will behave
the same.
> > >> > > >> >
> > >> > > >> > and for +1 votings, do you mean to separate Python/Gluon
API
> > >> > > versioning
> > >> > > >> as
> > >> > > >> > well?
> > >> > > >> >
> > >> > > >> > 2018-03-12 11:18 GMT-07:00 Naveen Swamy <mnnaveen@gmail.com
> >:
> > >> > > >> > > -1 for different versioning, it not only be
maintenance
> > >> nightmare
> > >> > > but
> > >> > > >> > also
> > >> > > >> > > more importantly confusing to users,
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > > On Mon, Mar 12, 2018 at 9:57 AM, Marco de
Abreu <
> > >> > > >> > > marco.g.abreu@googlemail.com> wrote:
> > >> > > >> > >
> > >> > > >> > >> According to the discussion in the Scala
thread, the
> release
> > >> > cycles
> > >> > > >> > would
> > >> > > >> > >> stay unchanged and are still part of the
mxnet releases.
> > >> > > >> > >>
> > >> > > >> > >> Nan Zhu <zhunanmcgill@gmail.com>
schrieb am Mo., 12. März
> > >> 2018,
> > >> > > >> 17:42:
> > >> > > >> > >>
> > >> > > >> > >> > how about release cycle?
> > >> > > >> > >> >
> > >> > > >> > >> >
> > >> > > >> > >> > On Mon, Mar 12, 2018 at 9:37 AM,
Yuan Tang <
> > >> > > terrytangyuan@gmail.com
> > >> > > >> >
> > >> > > >> > >> > wrote:
> > >> > > >> > >> >
> > >> > > >> > >> > > +1
> > >> > > >> > >> > >
> > >> > > >> > >> > > On Mon, Mar 12, 2018 at 12:35
PM, Marco de Abreu <
> > >> > > >> > >> > > marco.g.abreu@googlemail.com>
wrote:
> > >> > > >> > >> > >
> > >> > > >> > >> > > > +1
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > Tianqi Chen <tqchen@cs.washington.edu>
schrieb am
> > Mo.,
> > >> 12.
> > >> > > März
> > >> > > >> > >> 2018,
> > >> > > >> > >> > > > 17:33:
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > > +1
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > > On Mon, Mar 12, 2018
at 9:32 AM, Chris Olivier <
> > >> > > >> > >> > cjolivier01@apache.org
> > >> > > >> > >> > > >
> > >> > > >> > >> > > > > wrote:
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > > > > It has been proposed
that all Non-C API's follow
> > >> > separate
> > >> > > >> > >> > versioning
> > >> > > >> > >> > > > from
> > >> > > >> > >> > > > > > the main mxnet
C API/releases.
> > >> > > >> > >> > > > > >
> > >> > > >> > >> > > > > > A +1 vote is
in *favor of* using a different
> > >> versioning
> > >> > > for
> > >> > > >> > all
> > >> > > >> > >> > > > > > non-C-API's,
with each API (Scala, R, Julia,
> C++,
> > >> etc.)
> > >> > > >> having
> > >> > > >> > >> its
> > >> > > >> > >> > > own
> > >> > > >> > >> > > > > > version.
> > >> > > >> > >> > > > > >
> > >> > > >> > >> > > > > > A -1 vote is
*against* using a different
> > versioning
> > >> for
> > >> > > all
> > >> > > >> > >> > > > non-C-API's,
> > >> > > >> > >> > > > > > with all API's
(Scala, R, Julia, C++, etc.)
> > sharing
> > >> the
> > >> > > >> mxnet
> > >> > > >> > >> > > version.
> > >> > > >> > >> > > > > >
> > >> > > >> > >> > > > > > This vote will
conclude on Monday, March 19,
> 2018.
> > >> > > >> > >> > > > > >
> > >> > > >> > >> > > > > > Thanks,
> > >> > > >> > >> > > > > > -Chris
> > >> > > >> > >> > > > > >
> > >> > > >> > >> > > > >
> > >> > > >> > >> > > >
> > >> > > >> > >> > >
> > >> > > >> > >> >
> > >> > > >> > >>
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> > --
> > >> > > >> > Yizhi Liu
> > >> > > >> > DMLC member
> > >> > > >> > Amazon Web Services
> > >> > > >> > Vancouver, Canada
> > >> > > >> >
> > >> > > >>
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Yizhi Liu
> > >> > > DMLC member
> > >> > > Amazon Web Services
> > >> > > Vancouver, Canada
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Rahul Huilgol
> > >>
> >
> >
> >
> > --
> > Yizhi Liu
> > DMLC member
> > Amazon Web Services
> > Vancouver, Canada
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message