mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anirudh <anirudh2...@gmail.com>
Subject Re: [VOTE] Disconnect all non-C API's from mxnet versioning
Date Mon, 12 Mar 2018 23:30:43 GMT
-1
Because of the customer pain-points mentioned by others.
I think for good customer experience,
all code in MXNet repository excluding submodules and 3rd party dependencies
should map to the same version.

Anirudh

On Mon, Mar 12, 2018 at 4:17 PM, YiZhi Liu <eazhi.liu@gmail.com> wrote:

> Kellen, we are not talking about using wrong native package AFTER
> downloading the package. It's about deciding which version to use
> BEFORE downloading, and collecting information to debug.
>
> Copy-paste my previous words,
>
> "
> 1. It is harmful to user experience
>     1) Each time users want to use a specific feature, need to first
> check the mxnet core version, then check which frontend work with this
> core version.
>     2) Each time users have problem using a frontend (Scala/R/...)
> API, need to figure out which core version they are using.
>
> And as I describe in my #5. Imagine an inverse situation. When someone
> has a model trained by gluon 1.6.0, he want to deploy it to JVM, what
> Scala API version should he use? 1.6.0? No. And which R package
> version he should use? It is still different from either Gluon version
> or Scala API version. What a nightmare.
> "
>
> 2018-03-12 16:10 GMT-07:00 kellen sunderland <kellen.sunderland@gmail.com
> >:
> > @Rahul + Roshani, I would hear what you're saying if the user had to
> worry
> > about using the native package, but that worry is abstracted from them.
> > The scala package has a dependency on the native library and includes the
> > native lib inside the jar.  The correct lib is then bound against at
> > runtime.  I don't see how a user can use the wrong library or be confused
> > here unless the instructions on this page are incorrect:
> > https://github.com/apache/incubator-mxnet/tree/master/scala-package
> >
> > On Tue, Mar 13, 2018 at 12:02 AM, Rahul Huilgol <rahulhuilgol@gmail.com>
> > wrote:
> >
> >> -1 for the frontends having different versions than the backend. It not
> >> only creates confusion for new users, but also increases the work of
> >> developers who need to ensure compatibility. All this for a one-time
> change
> >> of namespace of a package?
> >>
> >> I think we should increase the major version number to make this change?
> >> Why do we have to 'wait' for 2.0? Who tells us that it's time for a 2.0
> >> version?
> >>
> >> I think expecting a user to look up version numbers on the website and
> >> ensure compatibility as suggested above, is not a simple task. Most
> users
> >> might not even know how the backend and frontend integrate. They might
> not
> >> even know that there is a C API which powers the frontends. Even knowing
> >> how to look up documentation for a particular version of MXNet is a
> >> non-trivial task right now. (And there are pages in a version's
> >> documentation which link to a file in another version). We should avoid
> >> introducing more complexity into the process. As developers we tend to
> >> overlook the important aspect of user experience. I think we should
> take a
> >> step back and look at this from the perspective of a user, not from
> that of
> >> a developer who works closely with MXNet.
> >>
> >> Regards,
> >> Rahul
> >>
> >> On Mon, Mar 12, 2018 at 3:12 PM, Roshani Nagmote <
> >> roshaninagmote2@gmail.com>
> >> wrote:
> >>
> >> > -1 for different versioning.
> >> >
> >> > I feel its just added confusion for users.
> >> >
> >> > On Mon, Mar 12, 2018 at 2:35 PM, YiZhi Liu <eazhi.liu@gmail.com>
> wrote:
> >> >
> >> > > Agree.
> >> > >
> >> > > And my reply to Marco's point,
> >> > >
> >> > > > Changing namespaces is one use-case, but there will be a lot
more
> >> with
> >> > > increasing activity - we have to take the bigger picture in mind.
> >> > > And you mentioned the CPP package as an example.
> >> > > > During analysis, we figured that a re-engineering of that API
> would
> >> be
> >> > > more appropriate and easier maintainable.
> >> > > I cannot agree as an engineer. Why not keep old API and add new
> ones?
> >> > > Just like in c_api.h, we added xxxEx while did not remove xxx, which
> >> > > keeps APIs compatible.
> >> > >
> >> > > > I think it is safe to say that the other APIs have not been
> >> maintained
> >> > > as actively as our Python/Gluon API.
> >> > > Are you saying, if an API is maintained actively and is widely used,
> >> > > then it should be versioned together with MXNet Core?
> >> > > Interesting, maybe instead we should have another discussion to
> decide
> >> > > whether to remove some of the 'inactive' frontend bindings from the
> >> > > repo.
> >> > >
> >> > > > We have to do #3 anyways, so it is just about having a different
> >> number
> >> > > set as version string.
> >> > > A release with 6 different versions and 5 mappings?
> >> > >
> >> > > > I really don't see an issue in #1 - it's a simple lookup that
> could
> >> be
> >> > > done on our website.
> >> > > Please be careful to say 'simple', each time we introduce an
> >> > > additional step, we lose a number of our potential users.
> >> > > And as I describe in my #5. Imagine an inverse situation. When
> someone
> >> > > has a model trained by gluon 1.6.0, he want to deploy it to JVM,
> what
> >> > > Scala API version should he use? 1.6.0? No. And which R package
> >> > > version he should use? It is still different from either Gluon
> version
> >> > > or Scala API version. What a nightmare.
> >> > >
> >> > > 2018-03-12 14:11 GMT-07:00 Chris Olivier <cjolivier01@gmail.com>:
> >> > > > Marco, you're mixing votes again.
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > * This leaves us with three options: 1. Vote failed: No
> refactoring
> >> of
> >> > > > user-facing APIs (including namespace changes) possible OR major
> >> > version
> >> > > > increase 2. Vote succeeded: Refactoring of user-facing APIs
> possible
> >> > and
> >> > > > only users of the changed APIs are affected while major version
> does
> >> > not
> >> > > > increase for other APIs. 3. Remove SemVer: We could introduce
> >> breaking
> >> > > > changes at any point in time, but our users would be losing trust
> due
> >> > to
> >> > > > unexpected failures during upgrades.*
> >> > > >
> >> > > > What you're describing is not what this vote is about.  This
vote
> is
> >> > > > whether to separate mxnet and API versioning.
> >> > > > Please try to stay on task.
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Mar 12, 2018 at 1:56 PM, Marco de Abreu <
> >> > > > marco.g.abreu@googlemail.com> wrote:
> >> > > >
> >> > > >> Regarding #4: Changing namespaces is one use-case, but there
will
> >> be a
> >> > > lot
> >> > > >> more with increasing activity - we have to take the bigger
> picture
> >> in
> >> > > mind.
> >> > > >> I think it is safe to say that the other APIs have not been
> >> maintained
> >> > > as
> >> > > >> actively as our Python/Gluon API (which I would say could
be
> >> versioned
> >> > > >> together with MXNet Core, but it does not really make a
> difference).
> >> > > This
> >> > > >> results in our APIs not reflecting all features available
in
> MXNet
> >> > (#2)
> >> > > or
> >> > > >> doing it in a way that we wouldn't recommend nowadays. While
it
> is
> >> no
> >> > > >> problem to add new features to an API using a minor version
> change,
> >> it
> >> > > >> limits our possibilites to do a refactor. Our team, for example,
> >> got a
> >> > > >> customer that would like to see the functionality of the
Cpp
> package
> >> > > being
> >> > > >> increased. During analysis, we figured that a re-engineering
of
> that
> >> > API
> >> > > >> would be more appropriate and easier maintainable. If we
don't
> pass
> >> > this
> >> > > >> vote, we won't be able to make any improvements to our less
> >> maintained
> >> > > APIs
> >> > > >> without a major version increment - which the community is
also
> >> > heavily
> >> > > >> against. We have to do #3 anyways, so it is just about having
a
> >> > > different
> >> > > >> number set as version string - right now we're making it
easy for
> >> > > ourselves
> >> > > >> by basically not maintaining any other than the Python interface
> and
> >> > > >> declining all breaking changes or refactors to APIs. I really
> don't
> >> > see
> >> > > an
> >> > > >> issue in #1 - it's a simple lookup that could be done on
our
> >> website.
> >> > > >> Simply select the version of MXNet you would like to have
and it
> >> will
> >> > > >> provide you with the appropriate installation instructions
- the
> >> same
> >> > > way
> >> > > >> we're already doing it.
> >> > > >>
> >> > > >> This leaves us with three options:
> >> > > >> 1. Vote failed: No refactoring of user-facing APIs (including
> >> > namespace
> >> > > >> changes) possible OR major version increase
> >> > > >> 2. Vote succeeded: Refactoring of user-facing APIs possible
and
> only
> >> > > users
> >> > > >> of the changed APIs are affected while major version does
not
> >> increase
> >> > > for
> >> > > >> other APIs.
> >> > > >> 3. Remove SemVer: We could introduce breaking changes at
any
> point
> >> in
> >> > > time,
> >> > > >> but our users would be losing trust due to unexpected failures
> >> during
> >> > > >> upgrades.
> >> > > >>
> >> > > >> -Marco
> >> > > >>
> >> > > >>
> >> > > >> On Mon, Mar 12, 2018 at 9:22 PM, YiZhi Liu <eazhi.liu@gmail.com>
> >> > wrote:
> >> > > >>
> >> > > >> > STRONGLY -1 (binding) as I explained in another thread
> 'Publishing
> >> > > >> > Scala Package/namespace change'. I think separating
version is
> >> > > >> > harmful.
> >> > > >> >
> >> > > >> > 1. It is harmful to user experience
> >> > > >> >     1) Each time users want to use a specific feature,
need to
> >> first
> >> > > >> > check the mxnet core version, then check which frontend
work
> with
> >> > this
> >> > > >> > core version.
> >> > > >> >     2) Each time users have problem using a frontend
> (Scala/R/...)
> >> > > >> > API, need to figure out which core version they are
using.
> >> > > >> > 2. Frontend APIs are tightly binding to the 'MXNet Core',
e.g.,
> >> > almost
> >> > > >> > all APIs extract operator definitions from the Core,
which
> makes
> >> the
> >> > > >> > API version and Core version a one-on-one mapping. Then
why
> >> > separate?
> >> > > >> > 3. It introduces overhead for release. Now each release
need to
> >> > > >> > involve a bunch of frontend release version, along with
the
> MXNet
> >> > core
> >> > > >> > release version.
> >> > > >> > 4. The only benefit I see so far is, it makes easier
for Scala
> >> > package
> >> > > >> > to change the namespace from ml.dmlc to org.apache (by
> increasing
> >> > > >> > Scala API major version id without changing the MXNet
core
> major
> >> > > >> > version). But,
> >> > > >> >    1) It is very likely that, this is the ONLY time
we benefit
> >> from
> >> > > >> > separate versioning. Changing namespace is a very rare
issue
> that
> >> > > >> > forces us to make APIs incompatible. In other situations,
the
> APIs
> >> > > >> > evolves smoothly which can stay compatible for a long
time.
> >> > > >> >    2) We can still discuss whether we have to change
the major
> >> > > version.
> >> > > >> >    3) Even the answer to 2) is Yes, I think it is affordable
to
> >> wait
> >> > > >> > for MXNet 2.0 to change 'ml.dmlc' to 'org.apache'
> >> > > >> > 5. Other Apache projects, e.g., Apache Spark, have PySpark
> (Python
> >> > > >> > frontend API), SparkR (R frontend API), MLLib, GraphX,
etc,
> same
> >> > > >> > version as the Spark Core, as well as the Scala/Java
API. I
> feel
> >> it
> >> > > >> > convenient since every time I check a document, say,
MLLib
> 1.6.0,
> >> I
> >> > > >> > can tell it works with Spark Core 1.6.0 and GraphX 1.6.0.
And I
> >> can
> >> > > >> > expect when I use Python API 1.6.0, it will behave the
same.
> >> > > >> >
> >> > > >> > and for +1 votings, do you mean to separate Python/Gluon
API
> >> > > versioning
> >> > > >> as
> >> > > >> > well?
> >> > > >> >
> >> > > >> > 2018-03-12 11:18 GMT-07:00 Naveen Swamy <mnnaveen@gmail.com>:
> >> > > >> > > -1 for different versioning, it not only be maintenance
> >> nightmare
> >> > > but
> >> > > >> > also
> >> > > >> > > more importantly confusing to users,
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > On Mon, Mar 12, 2018 at 9:57 AM, Marco de Abreu
<
> >> > > >> > > marco.g.abreu@googlemail.com> wrote:
> >> > > >> > >
> >> > > >> > >> According to the discussion in the Scala thread,
the release
> >> > cycles
> >> > > >> > would
> >> > > >> > >> stay unchanged and are still part of the mxnet
releases.
> >> > > >> > >>
> >> > > >> > >> Nan Zhu <zhunanmcgill@gmail.com> schrieb
am Mo., 12. März
> >> 2018,
> >> > > >> 17:42:
> >> > > >> > >>
> >> > > >> > >> > how about release cycle?
> >> > > >> > >> >
> >> > > >> > >> >
> >> > > >> > >> > On Mon, Mar 12, 2018 at 9:37 AM, Yuan
Tang <
> >> > > terrytangyuan@gmail.com
> >> > > >> >
> >> > > >> > >> > wrote:
> >> > > >> > >> >
> >> > > >> > >> > > +1
> >> > > >> > >> > >
> >> > > >> > >> > > On Mon, Mar 12, 2018 at 12:35 PM,
Marco de Abreu <
> >> > > >> > >> > > marco.g.abreu@googlemail.com>
wrote:
> >> > > >> > >> > >
> >> > > >> > >> > > > +1
> >> > > >> > >> > > >
> >> > > >> > >> > > > Tianqi Chen <tqchen@cs.washington.edu>
schrieb am
> Mo.,
> >> 12.
> >> > > März
> >> > > >> > >> 2018,
> >> > > >> > >> > > > 17:33:
> >> > > >> > >> > > >
> >> > > >> > >> > > > > +1
> >> > > >> > >> > > > >
> >> > > >> > >> > > > > On Mon, Mar 12, 2018 at
9:32 AM, Chris Olivier <
> >> > > >> > >> > cjolivier01@apache.org
> >> > > >> > >> > > >
> >> > > >> > >> > > > > wrote:
> >> > > >> > >> > > > >
> >> > > >> > >> > > > > > It has been proposed
that all Non-C API's follow
> >> > separate
> >> > > >> > >> > versioning
> >> > > >> > >> > > > from
> >> > > >> > >> > > > > > the main mxnet C API/releases.
> >> > > >> > >> > > > > >
> >> > > >> > >> > > > > > A +1 vote is in *favor
of* using a different
> >> versioning
> >> > > for
> >> > > >> > all
> >> > > >> > >> > > > > > non-C-API's, with
each API (Scala, R, Julia, C++,
> >> etc.)
> >> > > >> having
> >> > > >> > >> its
> >> > > >> > >> > > own
> >> > > >> > >> > > > > > version.
> >> > > >> > >> > > > > >
> >> > > >> > >> > > > > > A -1 vote is *against*
using a different
> versioning
> >> for
> >> > > all
> >> > > >> > >> > > > non-C-API's,
> >> > > >> > >> > > > > > with all API's (Scala,
R, Julia, C++, etc.)
> sharing
> >> the
> >> > > >> mxnet
> >> > > >> > >> > > version.
> >> > > >> > >> > > > > >
> >> > > >> > >> > > > > > This vote will conclude
on Monday, March 19, 2018.
> >> > > >> > >> > > > > >
> >> > > >> > >> > > > > > Thanks,
> >> > > >> > >> > > > > > -Chris
> >> > > >> > >> > > > > >
> >> > > >> > >> > > > >
> >> > > >> > >> > > >
> >> > > >> > >> > >
> >> > > >> > >> >
> >> > > >> > >>
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> > --
> >> > > >> > Yizhi Liu
> >> > > >> > DMLC member
> >> > > >> > Amazon Web Services
> >> > > >> > Vancouver, Canada
> >> > > >> >
> >> > > >>
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Yizhi Liu
> >> > > DMLC member
> >> > > Amazon Web Services
> >> > > Vancouver, Canada
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Rahul Huilgol
> >>
>
>
>
> --
> Yizhi Liu
> DMLC member
> Amazon Web Services
> Vancouver, Canada
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message