spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <>
Subject [jira] [Assigned] (SPARK-21688) performance improvement in mllib SVM with native BLAS
Date Mon, 28 Aug 2017 16:56:06 GMT


Apache Spark reassigned SPARK-21688:

    Assignee:     (was: Apache Spark)

> performance improvement in mllib SVM with native BLAS 
> ------------------------------------------------------
>                 Key: SPARK-21688
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>    Affects Versions: 2.2.0
>         Environment: 4 nodes: 1 master node, 3 worker nodes
> model name      : Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz
> Memory : 180G
> num of core per node: 10
>            Reporter: Vincent
>            Priority: Minor
>         Attachments: ddot unitest.png, mllib svm training.png, native-trywait.png, svm1.png,
svm2.png, svm-mkl-1.png, svm-mkl-2.png
> in current mllib SVM implementation, we found that the CPU is not fully utilized, one
reason is that f2j blas is set to be used in the HingeGradient computation. As we found out
earlier ( that with proper settings, native
blas is generally better than f2j on the uni-test level, here we make the blas operations
in SVM go with MKL blas and get an end to end performance report showing that in most cases
native blas outperformance f2j blas up to 50%.
> So, we suggest removing those f2j-fixed calling and going for native blas if available.
If this proposal is acceptable, we will move on to benchmark other algorithms impacted. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message