mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zach Kimberg <zachary.kimb...@gmail.com>
Subject Build from Source Instruction Changes
Date Wed, 28 Nov 2018 20:28:01 GMT
I would like to raise a PR for discussion here.  In the mxnet installation
docs, we currently have three inconsistent ways of compiling the mxnet
backend:

1. Use make by passing in the configuration directly

  make USE_BLAS=openblas

2. Use make with config.mk

  echo "USE_BLAS=openblas" >> ./config.mk

  make

3. Use cmake (which is only direct and does not work with config.mk)

  cmake USE_BLAS=open



I investigated this because we found that that passing configuration
directly can cause problems for building the scala frontend. The scala
frontend is compiled using make because it currently requires that the
building and linking flags that were used for building the backend also be
passed in to the frontend. Make computes the flags for both backend and
frontend, but if the make configurations differ between the two then it can
result in not using various libraries and subpar performance. The two ways
to fix that are either passing the same configuration "make
USE_BLAS=openblas; make scalapkg USE_BLAS=openblas" or that the
configuration be passed through config.mk (which unifies the
configuration). As it is far simpler, I opted for the config.mk solution.



However, there is also a movement to migrate from make to cmake. For this
reason, cmake has already begun appearing in some of the installation docs.
However, cmake does not use config.mk and has some differences in the
configurations from make.



The temporary fix that I implemented was to migrate all direct calls to
make (1) to uses of config.mk(2). I also take all the calls to build cmake
and redundantly add the flags to both config.mk and pass them directly to
cmake:

  echo "USE_BLAS=openblas" >> ./config.mk

  cmake USE_BLAS=open



You can see the specific changes in
https://github.com/apache/incubator-mxnet/pull/13364. There are also other
options for a temporary fix. We could remove all cmake usage for now and
then switch later. We could also switch entirely to cmake with redundant
config.mk flags.



The permanent fix would be to remove the scala compilation requirement for
the build flags. We are working on this, but it may take some time so we
want to make a temporary fix. Once it is done, then we will not have
problems with migrating to cmake. Because this affects the general
instructions, I want to post everything here in case there is some input.



Thanks,

Zach



- https://github.com/apache/incubator-mxnet/pull/13364

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message