spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "DB Tsai (JIRA)" <>
Subject [jira] [Created] (SPARK-4611) Implement the efficient vector norm
Date Wed, 26 Nov 2014 01:06:12 GMT
DB Tsai created SPARK-4611:

             Summary: Implement the efficient vector norm
                 Key: SPARK-4611
             Project: Spark
          Issue Type: Improvement
          Components: MLlib
            Reporter: DB Tsai

The vector norm in breeze is implemented by `activeIterator` which is known to be very slow.
In this PR, an efficient vector norm is implemented, and with this API, `Normalizer` has huge
performance gain.

Here is the benchmark against mnist8m dataset.


DenseVector: 68.25secs
SparseVector: 17.01secs

With this PR

DenseVector: 12.71secs
SparseVector: 2.73secs

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message