spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathaniel Wendt (JIRA)" <>
Subject [jira] [Commented] (SPARK-22619) Implement the CG method for ALS
Date Mon, 27 Nov 2017 19:05:00 GMT


Nathaniel Wendt commented on SPARK-22619:

I've implemented the algorithm and tested it's correctness but I am having trouble actually
seeing a performance speedup, likely due to incorrect handling of RDD persistence/checkpointing.
 I wasn't sure the best way to reach out to see if there were dev cycles available to collaborate
on completing this solution but I figure it has the potential to have a big impact within
Spark and MLLib.  If there is interest, I can open a pull request with the functionally correct
code I have as of now.

> Implement the CG method for ALS
> -------------------------------
>                 Key: SPARK-22619
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML
>    Affects Versions: 2.2.0
>            Reporter: Nathaniel Wendt
> The conjugate gradient method has been shown to be very efficient at solving the least
squares error problem in matrix factorization:
 Implementing this in Spark could mean a significant speedup in ALS solving as the order of
growth is smaller than the default solver (Cholesky).  This has the potential to improve the
training phase of collaborative filtering significantly.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message