spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vincent (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-21688) performance improvement in mllib SVM with native BLAS
Date Thu, 10 Aug 2017 08:14:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vincent updated SPARK-21688:
----------------------------
    Attachment:     (was: uni-test on ddot.png)

> performance improvement in mllib SVM with native BLAS 
> ------------------------------------------------------
>
>                 Key: SPARK-21688
>                 URL: https://issues.apache.org/jira/browse/SPARK-21688
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>    Affects Versions: 2.2.0
>         Environment: 4 nodes: 1 master node, 3 worker nodes
> model name      : Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz
> Memory : 180G
> num of core per node: 10
>            Reporter: Vincent
>         Attachments: ddot unitest.png, mllib svm training.png, svm1.png, svm2.png, svm-mkl-1.png,
svm-mkl-2.png
>
>
> in current mllib SVM implementation, we found that the CPU is not fully utilized, one
reason is that f2j blas is set to be used in the HingeGradient computation. As we found out
earlier (https://issues.apache.org/jira/browse/SPARK-21305) that with proper settings, native
blas is generally better than f2j on the uni-test level, here we make the blas operations
in SVM go with MKL blas and get an end to end performance report showing that in most cases
native blas outperformance f2j blas up to 50%.
> So, we suggest removing those f2j-fixed calling and going for native blas if available.
If this proposal is acceptable, we will move on to benchmark other algorithms impacted. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message