mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Larroy <pedro.larroy.li...@gmail.com>
Subject Re: using conan to manage Apache Incubator MXNet project dependencies
Date Tue, 27 Nov 2018 12:27:16 GMT
Hi Konstantin

Thanks for this contribution. With your proposed changes, when
building MXNet we will be pulling binaries for the libraries managed
by conan?


Pedro.
On Mon, Nov 26, 2018 at 11:43 AM Konstantin Ivlev <tomskside@gmail.com> wrote:
>
> Hello, Ivan,
>
> could you possibly clarify your question (may be explaining the use-case
> behind it)?
> Gradle appears to be build system, AFAIK more popular in Java world.
> meanwhile, Apache Incubator MXNet project uses CMake as its build system.
> please correct me, I am wrong.
> in general, conan, as a package manager, is pretty build-system-agnostic,
> and it may work with arbitrary build systems (to count a few, CMake,
> premake, qmake, full list:
> https://docs.conan.io/en/latest/reference/generators.html). I don't think
> Gradle is exception here.
> also, for instance, Android Studio also uses Gradle for Android C++
> projects, and conan works flawlessly in this particular case (
> https://blog.conan.io/2018/02/13/Android-Studio-project-Conan-Boost.html).
>
> yours sincerely, Konstantin
>
> пн, 26 нояб. 2018 г. в 16:43, Ivan Serdyuk <local.tourist.kiev@gmail.com>:
>
> > Kostantin, and what (overall) option with using Gradle? Does this your
> > suggested package manager has been supported by Gradle?
> >
> > Ivan
> >
> > On Mon, Nov 26, 2018 at 9:43 AM Konstantin Ivlev <tomskside@gmail.com>
> > wrote:
> >
> > > hello,
> > >
> > > this email is related to the following PR and JIRA ticket:
> > > - [MXNET-1229] use OpenBLAS, lapack & OpenCV from conan
> > > <https://github.com/apache/incubator-mxnet/pull/13400>
> > > - use conan to manage project dependencies
> > > <https://issues.apache.org/jira/browse/MXNET-1229>
> > >
> > > conan <https://conan.io> is an open-source package manager for C++
> > > projects. it allows to manage project dependencies in transparent and
> > > declarative manner.
> > >
> > > currently, apache incubator-mxnet project uses the following different
> > > ways to manage its dependencies:
> > >
> > > - download GitHub archives during the build
> > >     - OpenBLAS <https://github.com/xianyi/OpenBLAS>
> > >     - OpenCV <https://github.com/opencv/opencv>
> > > - conda <https://conda.io/> (alternative way to GitHub archives)
> > > - download from CMake
> > >     - Intel Math Kernel Library <https://software.intel.com/en-us/mkl>
> > > (MKL)
> > > - Git submodules
> > >     - cub <https://github.com/dmlc/cub>
> > >     - dlpack <https://github.com/dmlc/dlpack>
> > >     - dmlc-core <https://github.com/dmlc/dmlc-core>
> > >     - googletest <https://github.com/abseil/googletest>
> > >     - mkldnn <https://github.com/intel/mkl-dnn>
> > >     - mshadow <https://github.com/dmlc/mshadow>
> > >     - onnx-tensorrt <https://github.com/onnx/onnx-tensorrt>
> > >     - openmp <https://github.com/llvm-mirror/openmp>
> > >     - ps-lite <https://github.com/dmlc/ps-lite>
> > >     - tvm <https://github.com/dmlc/tvm>
> > >
> > > this appears to be very heterogeneous and hard to manage/maintain, as
> > > multiple various commands are in use to achieve dependencies
> > installation,
> > > as well as multiple places are to look for dependency versions and their
> > > updates.
> > >
> > > with conan, it may became much more straightforward, as dependencies will
> > > be declared in single place (conanfile) and installed via single command
> > > (conan install).
> > >
> > > as project is very complex, and has lots of dependencies, for the first
> > > prototype I've used only very few of dependencies from conan: OpenCV
> > > <https://bintray.com/conan-community/conan/opencv%3Aconan>, OpenBLAS
> > > <https://bintray.com/conan-community/conan/openblas%3Aconan> and lapack
> > > <https://bintray.com/conan-community/conan/lapack%3Aconan>.
> > > others may be easily added then one by one, but they first has to be
> > > packaged (not all of them are packaged yet, e.g. GoogleTest
> > > <https://bintray.com/bincrafters/public-conan/gtest%3Abincrafters> is
> > > available, while MKL is not).
> > >
> > > I attach patch which adds an initial conan support as proof of concept.
> > > also, I attach two simple build scripts, which I've used to test (for
> > > Windows and Linux / Mac OS X). Google Mail blocks .sh and .cmd
> > extensions,
> > > so you'll need to rename files.
> > > lemme know if you have any further questions.
> > >
> > > yours sincerely, Konstantin
> > >
> >

Mime
View raw message