mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YiZhi Liu <>
Subject Re: Publishing Scala Package/namespace change
Date Sun, 11 Mar 2018 11:16:14 GMT
I have no idea how separating Scala API version can solve the
'compatibility' problem. Do you mean we have different version for
'ml.dmlc' namespace and 'org.apache' namespace? Do these two versions
have same behavior? How to tell which Scala API version works with
which MXNet core version? By document? How many users will read the
whole document and carefully pair the version id before they run into
a strange error and give up?

Moreover, changing namespace is an issue that is really rare and
hardly happens. For other 'compatibility' problem, for example, the
class/function definitions, should handle the compatibility itself.
You'll never expect a project to have a different version for changing
'calculate(int)' to 'calculate(float)', it should just add a new
function 'calculate(float)'.

Regarding 'In this case the Scala interface is clearly a separate
entity from the C API.'. Everything can be seen as a separate entity,
the mxnet engine, the graph description, operators, python API, gluon
API, etc. We should think carefully what we want to provide, and what
our users need.

As an example, Apache Spark, still has SparkR (R API), PySpark (Python
API), MLLib, GraphX ... as part of its release, and have the same
version as Spark core as well as its Scala/Java API.

2018-03-10 23:58 GMT-08:00 kellen sunderland <>:
> +1 (non-binding) to what Marco is describing.  +1 (non-binding) to getting the Scala
bindings with the namespace change into Maven.
> The general best practice for SemVer, which is used by most projects that employ SemVer,
is to apply SemVer to the public APIs of packages that ship with your project.  If you have
several independent APIs this could mean that they are versioned separately from each other,
and from the overall project versioning mechanism.
> For example, the .NET Core library ships with a number of binaries, each with their own
SemVerioned APIs.  They also have a high-level, easy to understand version for the package
as a whole:
> Nodesource has a good description of this:
> “Semver is a scheme for interface versioning for the benefit of interface consumers,
thus if a tool has multiple interfaces, e.g. an API and a CLI, these interfaces may evolve
independent versioning.”
> SemVer at its core is a communication mechanism to inform developers of incompatibilities.
In this case the Scala interface is clearly a separate entity from the C API.  I.e. changing
the Scala namespace isn’t going to break C API users.  It does not communicate anything
useful to these users if we up their major version in response to a Scala change, it simply
breaks compatibility.  If we group all interfaces together, and increment whenever any of
them has a breaking change we’ll soon be at MXNet version 587.  We’ll be forcing our users
to check compatibility and update their dependency tracking constantly.  The end result is
that our users will stop pulling in new versions of the library.
> What I would propose is that (1) we have a high-level SemVer system that tracks our C_API.
 This is the ‘MXNet’ version that we generally refer to and emphasize for our public releases.
 For each API we have an independent versioning system that if we can, we fix to the MXNet
version.  When it makes sense we version these APIs independently.  So for example we could
have a MXNet 1.2 release that ships with a 2.0 Scala API / R API.
> In terms of Apache process I think shipping artifacts with a non-Apache namespace is
a bigger issue than whatever versioning conventions we decide to use.
> -Kellen
> From: Carin Meier
> Sent: Saturday, March 10, 2018 1:41 PM
> To:
> Cc:
> Subject: Re: Publishing Scala Package/namespace change
> +1 as well. I'm actively developing a Clojure package for MXNet that uses
> the jars from the Scala package.
> - Carin
> On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu <> wrote:
>> +1 for changing the namespace asap. for the maven deploy, we can have
>> it build along with pip deployment.
>> 2018-03-09 10:15 GMT-08:00 Naveen Swamy <>:
>> > Hi Guys,
>> >
>> > I am working on MXNet Scala Inference APIs
>> > <> along with another
>> > contributor Roshani. A while back I noticed that we haven't been
>> publishing
>> > the scala package to Maven for a while now(last one being v0.11.1a under
>> > the dmlc namespace).
>> > Currently users have to build the package manually and then use it, this
>> > hinders adoption and also is painful to build everything from source.
>> >
>> > I also see that we haven't changed the namespace to org.apache and
>> instead
>> > are still ml.dmlc namespace.
>> >
>> > I wanted to seek your opinion about changing the MXNet-Scala package
>> > namespace to org.apache for the Scala package and publish to Maven in the
>> > upcoming release. I understand that this probably breaks the Semver
>> > semantics that is agreed upon, However I would like to point out that the
>> > Scala package has never been published to maven as 1.0 under org.apache.
>> >
>> > Open to suggestions.
>> >
>> > Thanks, Naveen
>> --
>> Yizhi Liu
>> DMLC member
>> Amazon Web Services
>> Vancouver, Canada

Yizhi Liu
DMLC member
Amazon Web Services
Vancouver, Canada

View raw message