mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhao, Patric" <patric.z...@intel.com>
Subject RE: Performance regression from removing libiomp5.so
Date Thu, 12 Dec 2019 06:36:04 GMT
Thanks, Sam.

The root cause is from different OpenMP library. Intel OpenMP will provide better performance
as your data shown.

Regarding release, since the license issue[1], we can't ship Intel OpenMP in the binary, but
the most of performance boost from MKLDNN is still available.
I think it should be acceptable to release 1.6 with MKLDNN  + GNU OpenMP for suboptimal performance.


To achieve the best performance, user should build from source to enable more advanced features
like Intel MKL, Intel OpenMP, AVX512.

Thanks,

--Patric

[1] https://www.apache.org/legal/resolved.html#category-x



> -----Original Message-----
> From: Skalicky, Sam <sskalic@amazon.com.INVALID>
> Sent: Wednesday, December 11, 2019 1:36 PM
> To: dev@mxnet.incubator.apache.org
> Cc: Keshavan, Arjuna <arjunake@amazon.com>; Harish, Nihal
> <harnihal@amazon.com>
> Subject: Performance regression from removing libiomp5.so
> 
> Hi MXNet community,
> 
> I would like to bring your attention to the performance regression that was
> found [1] between 1.5.1 and 1.6.0 due to removing the libiomp5.so library
> due to licensing issues. This change was made since this library has a category
> x license [2] that is not compatible with the MXNet Apache
> license/distribution.
> 
> We found that using OpenBLAS instead of MKL BLAS caused a regression
> from 1500 samples/sec to 1300 samples/sec a 13.3% regression in training
> speed for a resnet18 training benchmark on a C5.18xlarge EC2 instance (with
> 72 cores). Rebuilding with MKL BLAS showed an increase in performance to
> 1600 samples/sec in the 1.6.0 branch.
> 
> Please provide your feedback on the licensing issue (are there any work-
> arounds) and the tradeoff in performance (is the benefit worth trying to
> include back into MXNet builds).
> 
> Thanks to the efforts of the following folks for working on this issue (in no
> particular order):
> Patric Zhao
> Amol Lele
> Tao Lv A
> Pedro Larroy
> Nihal Harish
> Chai Bapat
> Arjuna Keshavan
> Rong Zhang
> 
> Thanks!
> Sam
> 
> [1] https://github.com/apache/incubator-mxnet/issues/16891
> [2] https://www.apache.org/legal/resolved.html#category-x

Mime
View raw message