mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Re: IRStats Evaluation for Recommender Systems
Date Thu, 30 May 2013 19:01:44 GMT
THere's nothing direct, but you can probably save yourself time by copying
the code that computes these stats and apply them to your pre-computed
values. It's not terribly complex, just counting the intersection and union
size and deriving some stats from it.

The split is actually based on value -- higher-valued items are held out.
This has problems, but different ones from holding out newest data (i.e.
what if new ratings are all very negative? you don't want to have
recommended them). But both are sensible. There is a hook
called RelevantItemsDataSplitter that lets you define how the split is
computed though.

Sean


On Thu, May 30, 2013 at 1:14 PM, Parimi Rohit <rohit.parimi@gmail.com>wrote:

> Hi All,
>
> Is there a way to compute precision and recall values given a file of
> recommendations and a test file of user preferences.
>
> I know there is "GenericRecommenderIRStatsEvaluator" in Mahout to compute
> the IR Stats but it takes a "RecommenderBuilder" object among others as
> parameters to build a recommender and compute these metrics. However, if I
> already have a file of recommendations and a test file of preferences, I
> will not be able to use this class.
>
> Another use-case is when my data is temporal i.e I use past data for about
> a month to train my model and test the recommendations using 1 week future
> data (backtesting framework). I will not be able to use the above class as
> it splits the data randomly (I may be wrong in this case).
>
> To Summarize, I would like to compute the IR stats for a file of
> recommendations and a test file of use preferences and would like to know
> if this can be done using some class in Mahout.
>
> Any help is much appreciated.
>
> Thanks,
> Rohit
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message