spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From BenFradet <...@git.apache.org>
Subject [GitHub] spark pull request: [SPARK-12247] [ML] [DOC] Documentation for spa...
Date Sat, 13 Feb 2016 17:18:40 GMT
Github user BenFradet commented on a diff in the pull request:

    https://github.com/apache/spark/pull/10411#discussion_r52830587
  
    --- Diff: docs/ml-collaborative-filtering.md ---
    @@ -0,0 +1,147 @@
    +---
    +layout: global
    +title: Collaborative Filtering - spark.ml
    +displayTitle: Collaborative Filtering - spark.ml
    +---
    +
    +* Table of contents
    +{:toc}
    +
    +## Collaborative filtering 
    +
    +[Collaborative filtering](http://en.wikipedia.org/wiki/Recommender_system#Collaborative_filtering)
    +is commonly used for recommender systems.  These techniques aim to fill in the
    +missing entries of a user-item association matrix.  `spark.ml` currently supports
    +model-based collaborative filtering, in which users and products are described
    +by a small set of latent factors that can be used to predict missing entries.
    +`spark.ml` uses the [alternating least squares
    +(ALS)](http://dl.acm.org/citation.cfm?id=1608614)
    +algorithm to learn these latent factors. The implementation in `spark.ml` has the
    +following parameters:
    +
    +* *numBlocks* is the number of blocks the users and items will be partitioned into in
order to parallelize computation (defaults to 10).
    +* *rank* is the number of latent factors in the model (defaults to 10).
    +* *maxIter* is the maximum number of iterations to run (defaults to 10).
    +* *regParam* specifies the regularization parameter in ALS (defaults to 1.0).
    +* *implicitPrefs* specifies whether to use the *explicit feedback* ALS variant or one
adapted for
    +  *implicit feedback* data (defaults to `false` which means using *explicit feedback*).
    +* *alpha* is a parameter applicable to the implicit feedback variant of ALS that governs
the
    +  *baseline* confidence in preference observations (defaults to 1.0).
    +* *nonnegative* specifies whether or not to use nonnegative constraints for least squares
(defaults to `false`).
    +
    +### Explicit vs. implicit feedback
    +
    +The standard approach to matrix factorization based collaborative filtering treats 
    +the entries in the user-item matrix as *explicit* preferences given by the user to the
item.
    +For example, users giving ratings to movies.
    +
    +It is common in many real-world use cases to only have access to *implicit feedback*
(e.g. views, 
    +clicks, purchases, likes, shares etc.). The approach used in `spark.mllib` to deal with
such data is taken
    +from [Collaborative Filtering for Implicit Feedback Datasets](http://dx.doi.org/10.1109/ICDM.2008.22).
    +Essentially, instead of trying to model the matrix of ratings directly, this approach
treats the data
    --- End diff --
    
    @srowen tried to take your remarks into account, I don't know if it's clearer now though.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message