mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sheng Zha <zhash...@apache.org>
Subject Re: [VOTE] add conan support for Apache MXNet (incubating)
Date Sat, 04 May 2019 23:25:27 GMT
To be clear, my intention is really to prevent a seemingly good solution to exacerbate the
problem that it sets out to solve. This tends to happen when there are not enough people to
drive it to the end.

If there are additional values in this solution that people feel outweighs the problems below,
I'd be more than happy to be persuaded to vote otherwise.

-sz

On 2019/05/04 23:08:43, Sheng Zha <zhasheng@apache.org> wrote: 
> Thank you for the explanation and sorry that I missed the earlier context as it has been
a while. While I like the idea of simplifying the dependency management with tools like conan,
I have the following concerns on this vote as-is (it's also my take on why I think the PR
is stuck):
> 
> - It's unclear how much dependency needs can conan help in mxnet builds.
>   Currently, the linked PR only includes OpenBLAS. A proof-of-concept that shows it actually
replaces more dependency than openblas would be helpful. On a high-level, there are two types
of builds for mxnet:
>   * User's custom build-from-source: 1) usually dynamic linking is used. 2) users may
not enable all features, and users may want to pull a subset of the dependencies. 3) users
may want mxnet build system to pull the dependencies, or they may not. (for conan it's ok
to just focus on the former)
>   * Binary distribution for pip and maven: 1) static linking is used (requires -fPIC).
2) all features are enabled. 3) dependencies are pulled in with scripts in mxnet.
>   Handling one of the above cases would be a good showcase for the value of the new tool.
> 
> - It's unclear how it impacts people with only an archive of mxnet source code but without
network access.
>   This applies for the dependencies that are captured as submodules that you mentioned
as a way that mxnet manages dependency.
> 
> - If the value proposition of conan is to simplify the dependency management, then it
should unify other solutions instead of keeping these solutions around.
> 
> Overall, it would be helpful to have a clear message such as what exactly conan can replace,
and having a proof of concept that works for this would be helpful. Otherwise, I fear that
we may be introducing yet another way to manage dependency that further complicates the existing
problem.
> 
> That said, I'm not suggesting that we impose the burden to implement everything on you
alone, and it's ok to rally people who are interested in this solution to help out. To facilitate
this, I created a feature branch so that it's easier for you and people who are enthusiastic
about this to work together [1].
> 
> For now, I'm voting -1 to this proposal and I hope you understand.
> 
> -sz
> 
> [1] https://github.com/apache/incubator-mxnet/tree/conan
> 
> On 2019/05/03 07:51:34, Konstantin Ivlev <tomskside@gmail.com> wrote: 
> > hi Sheng Zha,
> > 
> > on pull request review I was told by Anirudh anirudhacharya and Roshani
> > Nagmote to start discussion/vote on the mxnet dev list. it seems to be a
> > vicious circle now - on GitHub I am told to use vote, and on vote I am told
> > to use GitHub, this doesn't help much.
> > FYI GitHub review stuck, it's already opened since November 2018, and it's
> > still not approved (however, there were no objections during the review).
> > Previous discussion in e-mail thread also didn't encounter any objections,
> > and all questions were answered.
> > JIRA ticket has no discussion at all (except it has duplicates of comments
> > made on GitHub).
> > so let's process with 3-day vote for now, as other communication channels
> > were already tried with no success.
> > 
> > yours sincerely, Konstantin
> > 
> > пт, 3 мая 2019 г. в 14:17, Sheng Zha <zhasheng@apache.org>:
> > 
> > > Hi Konstantin,
> > >
> > > While conan looks like an option that's worth exploring, given that your
> > > request is to merge the pull request, I'd suggest that the request should
> > > go through the regular pull request review and it doesn't really need a
> > > vote (as it doesn't substitute reviews anyway)
> > >
> > > If you would like to gather more attention to it, feel free to ping in a
> > > discussion thread.
> > >
> > > -sz
> > >
> > > On 2019/05/03 06:29:55, Konstantin Ivlev <tomskside@gmail.com> wrote:
> > > > Dear MXNet community,
> > > >
> > > > This is the 3-day vote to add conan support for Apache MXNet (incubating)
> > > > version v1.4.1.
> > > > The voting on dev@ list will start May 03 23:59:59 (PST) and close on
> > > May
> > > > 06 23:59:59.
> > > >
> > > > Background: conan is open-source, freeware, cross-platform package
> > > manager
> > > > for C and C++ projects, written in python. it provides integration with
> > > > various build systems, include CMake. conan may use bintray as a server
> > > to
> > > > store and download pre-built packages, or packages might be always built
> > > > from sources.
> > > >
> > > > Problem: currently (as for v1.4.1), Apache MXNet (incubating) is using
> > > > several ways to fetch 3rd-party dependencies simultaneously, for
> > > instance:
> > > > 1. download GitHub archives during the build
> > > > - OpenBLAS
> > > > - OpenCV
> > > > 2. conda (alternative way to GitHub archives)
> > > > 3. download from CMake
> > > > - Intel Math Kernel Library (MKL)
> > > > 4. Git submodules
> > > > - cub
> > > > - dlpack
> > > > - dmlc-core
> > > > - googletest
> > > > - mkldnn
> > > > - mshadow
> > > > - onnx-tensorrt
> > > > - openmp
> > > > - ps-lite
> > > > - tvm
> > > > therefore, there are multiple places to look for 3rd parties, and its
> > > hard
> > > > to update them, as you need to remember or figure it out how to update
a
> > > > particular dependency to newer version, for instance.
> > > > current Apache MXNet (incubating) build instructions differ very much
per
> > > > platform, require to download and unzip some archives manually,
> > > specifying
> > > > variables with paths to this archives, in conjunction of updating git
> > > > submodules,
> > > >
> > > > Action: merge pull request providing an initial conan support for Apache
> > > > MXNet (incubating). support conan as an alternate approach to fetch
> > > various
> > > > 3rd-party dependencies. old approaches will be still available, supported
> > > > and left intact.
> > > >
> > > > Below are links to
> > > > 1) conan web-site:  https://conan.io/
> > > > 2) conan GitHub repository: https://github.com/conan-io/conan
> > > > 3) conan documentation: https://docs.conan.io/en/latest/
> > > > 4) bintray: https://bintray.com
> > > > 5) pull request adding conan support to Apache MXNet (incubating):
> > > > https://github.com/apache/incubator-mxnet/pull/13400
> > > > 6) JIRA issue: https://issues.apache.org/jira/browse/MXNET-1229
> > > > 7) previous email discussion:
> > > >
> > > https://lists.apache.org/thread.html/301a46a637f7e3c249c475713f701bef7530c32bc92d8834c0882897@%3Cdev.mxnet.apache.org%3E
> > > > 8) MXNet build instructions:
> > > > https://mxnet-tqchen.readthedocs.io/en/latest/how_to/build.html
> > > > 9) MXNet build instructions (Windows):
> > > >
> > > https://mxnet.incubator.apache.org/versions/master/install/windows_setup.html
> > > > 10) MXNet build instructions (OSX):
> > > > http://mxnet.incubator.apache.org/versions/master/install/osx_setup.html
> > > > 11) MXNet build instructions (Linux):
> > > >
> > > http://mxnet.incubator.apache.org/versions/master/install/ubuntu_setup.html
> > > > 12) MXNet development setup (OSX):
> > > >
> > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+Developer+Setup+on+Mac
> > > >
> > > > Please remember to TEST first before voting accordingly:
> > > > +1 = approve
> > > > +0 = no opinion
> > > > -1 = disapprove (provide reason)
> > > >
> > > > Best regards,
> > > > Konstantin Ivlev
> > > >
> > >
> > 
> 

Mime
View raw message