spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject git commit: [SPARK-2843][MLLIB] add a section about regularization parameter in ALS
Date Thu, 21 Aug 2014 00:47:46 GMT
Repository: spark
Updated Branches:
  refs/heads/master e1571874f -> e0f946265

[SPARK-2843][MLLIB] add a section about regularization parameter in ALS

atalwalkar srowen

Author: Xiangrui Meng <>

Closes #2064 from mengxr/als-doc and squashes the following commits:

b2e20ab [Xiangrui Meng] introduced -> discussed
98abdd7 [Xiangrui Meng] add reference
339bd08 [Xiangrui Meng] add a section about regularization parameter in ALS


Branch: refs/heads/master
Commit: e0f946265b9ea5bc48849cf7794c2c03d5e29fba
Parents: e157187
Author: Xiangrui Meng <>
Authored: Wed Aug 20 17:47:39 2014 -0700
Committer: Xiangrui Meng <>
Committed: Wed Aug 20 17:47:39 2014 -0700

 docs/ | 11 +++++++++++
 1 file changed, 11 insertions(+)
diff --git a/docs/ b/docs/
index ab10b2f..d5c539d 100644
--- a/docs/
+++ b/docs/
@@ -43,6 +43,17 @@ level of confidence in observed user preferences, rather than explicit
ratings g
 model then tries to find latent factors that can be used to predict the expected preference
of a
 user for an item.
+### Scaling of the regularization parameter
+Since v1.1, we scale the regularization parameter `lambda` in solving each least squares
problem by
+the number of ratings the user generated in updating user factors,
+or the number of ratings the product received in updating product factors.
+This approach is named "ALS-WR" and discussed in the paper
+"[Large-Scale Parallel Collaborative Filtering for the Netflix Prize](".
+It makes `lambda` less dependent on the scale of the dataset.
+So we can apply the best parameter learned from a sampled subset to the full dataset
+and expect similar performance.
 ## Examples
 <div class="codetabs">

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message