mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ziheng Jiang <zih...@apache.org>
Subject Re: Apache MXNet Scala nightly-build now available on Apache Nexus
Date Mon, 29 Oct 2018 19:05:19 GMT
Nice work! It is quite helpful for deployment of MXNet Scala!

On Mon, Oct 29, 2018 at 10:39 AM YiZhi Liu <liuyizhi@apache.org> wrote:

> Hi,
>
>
>
> I am happy to announce that the availability of the experimental
> nightly-build Scala package on Maven in Nexus. The nightly-builds,
> currently published as 1.3.1-SNAPSHOT version, include the latest changes
> from the master branch from apache/incubator-mxnet GitHub repo and refresh
> every day. Furthermore, we completely overhauled the MXNet library shipped
> inside and adopted the binary build logic used for Python pip, so now Scala
> users can also enjoy the portable MXNet with fully loaded features.
> Currently, the supported artifacts are:
>
> - mxnet-full_2.11-linux-x86_64-cpu (Linux-CPU)
>
> - mxnet-full_2.11-linux-x86_64-gpu (Linux-GPU, CUDA 9.0)
>
> - mxnet-full_2.11-osx-x86_64-cpu (OSX-CPU)
>
>
>
> FAQ:
>
> - What's new in the nightly Scala packages compared to the previous Scala
> Maven artifacts?
>
> Most notably, better usability, more supported features, and better
> platform compatibility. The new packages now support Ubuntu 14.04 and
> Amazon Linux and support the same broad set of features as the existing pip
> packages. Besides, users are no longer required to install all the
> dependencies to use the nightly Scala packages as the included MXNet
> libraries are fully loaded. For the list of supported platforms and
> features, see [1].
>
>
>
> - What's different in the build approach compared to what's in the previous
> Scala Maven packages?
>
> The previous Scala Maven packages (1.2, 1.2.1, 1.3) were built with the
> same build commands that regular users would use when building from source
> [2]. This naive approach limits the compatibility to only similar
> environments as where the packages were built (which was Ubuntu 16.04 for
> Linux, and OSX 10.13 for OSX). As a result, users cannot use it on more
> CentOS or Amazon Linux and may suffer the problem of mismatching of
> dependency versions due to the difference in their environment.
>
> On the other hand, Python users have long been enjoying portable and fully
> loaded MXNet packages from pip. The MXNet libraries in pip packages are
> built in more controlled environments with compatibility in mind.
> Dependencies for extended features are statically linked and masked so that
> no additional steps are needed.
>
> Now, in the new Scala nightly packages, the same approach as pip is used
> for building the MXNet library so that the Scala users can benefit from the
> same great features.
>
>
>
> - How do I get the new nightly packages?
>
> You can find the packages on Apache (Snapshots) repository [3] including
> pom files and jars, which you can download and add to your projects.
> Alternatively, you can configure your pom.xml to use the Maven repository.
> Remember to add repository config:
>
> <repositories>
>
>   <repository>
>
>     <id>Apache Snapshot</id>
>
>     <url>https://repository.apache.org/content/groups/snapshots</url>
>
>   </repository>
>
> </repositories>
>
>
>
> - What's the support level of the Scala nightly packages?
>
> This package is currently experimental, so use at your own risk. While the
> first nightly packages are already manually verified on several platforms,
> and the automated publish pipeline runs unit tests and integration tests on
> OSX and Ubuntu 14.04 before publishing, the testing is currently limited.
>
>
>
> - How can I help?
>
> We call on all MXNet Scala community to test out the new nightly packages.
>
>
>
> - I found a problem in using the package. What should I do?
>
> Please report this in the Github issue in [1].
>
>
>
> - What's next?
>
> There are several more things to be done:
>
> 1. We are adding more comprehensive support for GPU versions for different
> CUDA versions.
>
> 2. We are adding more tests on more platforms. We are currently examining
> the MXNet's Jenkins CI solution for supporting more platforms.
>
> 3. As we recognize that the build logic benefits the community, we plan to
> open source the build scripts in the mxnet main repo. See progress at [5].
>
>
>
> Qing Lan, Zach Kimberg, Sheng Zha, and I worked together to make this
> happen. Thanks to Sheng Zha for sharing the pip build logic, and for
> building the automated publishing pipelines. Special thanks to Qing and
> Zach for sharing their expertise in Scala with Sheng and guiding and
> supporting him locally for this milestone.
>
>
>
> [1] https://github.com/apache/incubator-mxnet/issues/8671
>
> [2]
>
> https://github.com/apache/incubator-mxnet/blob/master/scala-package/dev/compile-mxnet-backend.sh#L59-L105
>
> [3]
>
> https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.3.1-SNAPSHOT~~
>
> [4] https://maven.apache.org/repository-management.html
>
> [5] https://github.com/apache/incubator-mxnet/projects/13
>
-- 
Ziheng Jiang
Fudan University | Computer Science

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message