spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "DB Tsai (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-4611) Implement the efficient vector norm
Date Wed, 26 Nov 2014 01:06:12 GMT
DB Tsai created SPARK-4611:
------------------------------

             Summary: Implement the efficient vector norm
                 Key: SPARK-4611
                 URL: https://issues.apache.org/jira/browse/SPARK-4611
             Project: Spark
          Issue Type: Improvement
          Components: MLlib
            Reporter: DB Tsai


The vector norm in breeze is implemented by `activeIterator` which is known to be very slow.
In this PR, an efficient vector norm is implemented, and with this API, `Normalizer` has huge
performance gain.

Here is the benchmark against mnist8m dataset.

Before

DenseVector: 68.25secs
SparseVector: 17.01secs

With this PR

DenseVector: 12.71secs
SparseVector: 2.73secs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message