mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lausen, Leonard" <lau...@amazon.com.INVALID>
Subject jemalloc 5 incompatibility
Date Sun, 19 Jan 2020 10:19:58 GMT
As of jemalloc 5, jemalloc default build can not be used in libraries that are
dlopened. However, libmxnet.so is dlopened by Python (ctypes). To use MXNet with
jemalloc 5, users must not link to system libjemalloc.so but must rather link to
a libjemalloc compiled with special parameters to allow dlopen to work. See 
https://github.com/jemalloc/jemalloc/issues/937

jemalloc 5 is distributed as part of Ubuntu 18.10 and higher, as well as Debian
Stable. Users on these systems will be unable to use MXNet after compiling with
USE_JEMALLOC=ON on systems with libjemalloc-dev, which is the default setting.

Thus in https://github.com/apache/incubator-mxnet/pull/17324 I suggest to
disable Jemalloc by default in the source build. Auto-detecting the version of
jemalloc is not helpful, because over time less and less systems will come with
a working version of jemalloc.

Please go ahead and approve the PR if you agree.

The better solution is to build jemalloc as part of our build. See 
https://github.com/apache/incubator-mxnet/pull/17121
But as CMake build of jemalloc is not yet integrated upstream, this approach
currently relies on a development branch of jemalloc. Thus it was suggested to
revisit the approach once CMake integration is stable upstream.
Mime
View raw message